Xen3 and a Virtual Network
Contents
* 1 Introduction
* 2 A Virtual Network
* 3 The Brouter Setup
* 4 The Configuration
* 5 The Script
* 6 The Firewall Setup
Introduction
By default Xen creates a bridge to which it attaches a physical network interface. To keep the host or priviledged system (dom0) working, it creates a virtual interface for it and attaches this to the bridge.
This works quite transparently on a workstation but has issues on a server which also serves as a router and firewall. Standard answer, and best practice according to other Xen users, is to hide the physical network interfaces for dom0 and expose them in a domU and have it handle the routing and firewalling.
A Virtual Network
For my purpose, I would like to run a Xen guest incidentally on my SOHO server. This has some implications, especially when using a firewall. Packets coming from a standard bridge pass the firewall tables differently and will never be masqueraded. Unless you hack extra rules using ebtables which is what iptables is for routed traffic.
My requirements were actually not to have a physical nic renumbered, renamed, put in promiscuous mode and attached to a bridge, but to have a virtual network on my server which will be treated as a dmz.
The Brouter Setup
To accomplish this, and to have the domU's on my dmz masqueraded to the net, I tried a brouter setup. And it works wonderfully well.
A brouter is a combination of a bridge an a router. In this configuration the interfaces attached to it do not have an ip address, but the bridge does. Sticking with the descriptions used in the Xen manual, it is a switch and network interface in one, attached to dom0 and to which the domU's are connected.
So the domU's are bridged to each other, but the connection to dom0 is routed. Hence the name 'brouter'.
It looks like this:
Image:XenWithVirtualNetwork480.png
The startup procedure is in essence as follows:
1. At the start of Xend
* Start a bridge
* Configure it with an ip address as if it was a normal network interface
* Set the proper route to it and restart the firewall.
2. At the start of the domU
* Attach the vif to the bridge.
* Start the domU
To accomplish this I use a special script to be started when xend is started, but still use the regular (bridging) script for starting the domU's.
The Configuration
my /etc/xen/xend-config.sxp has contains the following settings for the network (disable any others):
## Use the following if the domU's are on their own virtual net
# and dom0 does all routing etc. This is actually a brouter
(network-script 'network-virtual bridgeip="10.98.98.1/24" brnet="10.98.98.0/24"')
(vif-script vif-bridge)
and my domU configuration looks as follows:
disk = [ 'file:/var/lib/xen/images/MyDomU/hda,hda1,w', 'file:/var/lib/xen/images/MyDomU/swap,hda2,w']
memory = 128
kernel = "/var/lib/xen/images/MyDomU/vmlinuz-xen"
ramdisk = "/var/lib/xen/images/MyDomU/initrd-xen"
name = "MyDomU"
root = "/dev/hda1"
vif = ['mac=aa:cc:00:00:00:01']
You will notice the absense of any network settings in this configuration file. That is because I use the regular SUSE configuration system in my domU to set these.
The Script
Maybe it would be logical to show the script before the configuratione, but due to it's size I thought it better to put it at the end of the page.
The script is called network-virtual and should be located in /etc/xen/scripts
#!/bin/sh
#============================================================================
# Default Xen network start/stop script.
# Xend calls a network script when it starts.
# The script name to use is defined in /etc/xen/xend-config.sxp
# in the network-script field.
#
# This script creates a bridge (default xenbr${vifnum}), gives it an IP address
# and the appropriate route. Then it starts the SuSEfirewall2 which should have
# the bridge device in the zone you want it.
#
# If all goes well, this should ensure that networking stays up.
# However, some configurations are upset by this, especially
# NFS roots. If the bridged setup does not meet your needs,
# configure a different script, for example using routing instead.
#
# Usage:
#
# vnet-brouter (start|stop|status) {VAR=VAL}*
#
# Vars:
#
# bridgeip Holds the ip address the bridge should have in the
# the form ip/mask (10.0.0.1/24).
# brnet Holds the network of the bridge (10.0.0.1/24).
#
# vifnum Virtual device number to use (default 0). Numbers >=8
# require the netback driver to have nloopbacks set to a
# higher value than its default of 8.
# bridge The bridge to use (default xenbr${vifnum}).
#
# start:
# Creates the bridge
# Gives it the IP address and netmask
# Adds the routes to the routing table.
#
# stop:
# Removes all routes from the bridge
# Removes any devices on the bridge from it.
# Deletes bridge
#
# status:
# Print addresses, interfaces, routes
#
#============================================================================
dir=$(dirname "$0")
. "$dir/xen-script-common.sh"
. "$dir/xen-network-common.sh"
findCommand "$@"
evalVariables "$@"
vifnum=${vifnum:-0}
bridgeip=${bridgeip:-10.6.7.1/24}
brnet=${brnet:-10.6.7.0/24}
netmask=${netmask:-255.255.255.0}
bridge=${bridge:-xenbr${vifnum}}
##
# link_exists interface
#
# Returns 0 if the interface named exists (whether up or down), 1 otherwise.
#
link_exists()
{
if ip link show "$1" >/dev/null 2>/dev/null
then
return 0
else
return 1
fi
}
# Usage: create_bridge bridge
create_bridge () {
local bridge=$1
# Don't create the bridge if it already exists.
if [ ! -d "/sys/class/net/${bridge}/bridge" ]; then
brctl addbr ${bridge}
brctl stp ${bridge} off
brctl setfd ${bridge} 0
fi
ip link set ${bridge} up
}
# Usage: add_to_bridge bridge dev
add_to_bridge () {
local bridge=$1
local dev=$2
# Don't add $dev to $bridge if it's already on a bridge.
if ! brctl show | grep -wq ${dev} ; then
brctl addif ${bridge} ${dev}
fi
}
# Usage: show_status dev bridge
# Print ifconfig and routes.
show_status () {
local dev=$1
local bridge=$2
echo '============================================================'
ip addr show ${dev}
ip addr show ${bridge}
echo ' '
brctl show ${bridge}
echo ' '
ip route list
echo ' '
route -n
echo '============================================================'
}
op_start () {
if [ "${bridge}" = "null" ] ; then
return
fi
create_bridge ${bridge}
if link_exists "$bridge"; then
ip address add dev $bridge $bridgeip
ip link set ${bridge} up arp on
ip route add to $brnet dev $bridge
fi
if [ ${antispoof} = 'yes' ] ; then
antispoofing
fi
rcSuSEfirewall2 start
}
op_stop () {
if [ "${bridge}" = "null" ]; then
return
fi
if ! link_exists "$bridge"; then
return
fi
ip route del to $brnet dev $bridge
ip link set ${bridge} down arp off
ip address del dev $bridge $bridgeip
##FIXME: disconnect the interfaces from the bridge 1st
brctl delbr ${bridge}
rcSuSEfirewall2 start
}
case "$command" in
start)
op_start
;;
stop)
op_stop
;;
status)
show_status ${netdev} ${bridge}
;;
*)
echo "Unknown command: $command" >&2
echo 'Valid commands are: start, stop, status' >&2
exit 1
esac
The Firewall Setup
Moving forward in the same direction, I wrote a similar script to handle Firewall (Filter and Forward) rules in a Xen environment. It allows you to handle port forward from external NIC interface(s) handle by Dom0 to the virtual NIC as seen by DOM-1...Dom-N.
The target is to reproduce a somehow equivalent network environment of a traditional ISP, but in a virtual XEN world. Allowing people to defines network zones (DMZ,HOSTING,DATA, ...) with adequate firewall rules on how packets move from the external layer to the internal one.
Full description here:
