Gentoo xen dom0/domU nano howto

Gentoo xen dom0/domU nano howto
Table of Contents

    1 Prerequiresites
    2 Setting up tools
    3 dom0 configuration
        3.1 Xen configuration
        3.2 Network/bridge configuration
        3.3 Configure boot loaders
    4 domU configuration
    5 Start domU
        5.1 Create domU launcher script:
        5.2 Start xen services
        5.3 Launch domU
    6 Misc

1 Prerequiresites

    Kernel version: 3.1+ Kernel 3.1 have xen dom0 support, so we don't have to build xen dom0 kernel.
    Xen version: 4.1+

2 Setting up tools

echo "app-emulation/xen-tools            hvm qemu screen xend" >> /etc/portage/package.use

emerge -av xen-tools xen xen-pvgrub

3 dom0 configuration
3.1 Xen configuration

edit /etc/xen/xend-config.sxp, make sure the belows lines are present:

(network-script network-bridge)
...
(vif-script vif-bridge bridge=xenbr0)

3.2 Network/bridge configuration

Please also note bridge xenbr0 should be the bridge interface being used. You need create this interface if you don't have. IE: by modify /etc/conf.d/net:

config_eth0=( "null" )

bridge_xenbr0="eth0"
config_xenbr0=( "192.168.100.123/24" )
RC_NEED_xenbr0="net.eth0"
brctl_xenbr0=( "setfd 0" "sethello 1" "stp off" )
routes_xenbr0=( "default gw 192.168.100.1" )

And add xenbr0 to start script like:

cd /etc/init.d/
ln -s net.lo net.xenbr0
rc-update add net.xenbr0 default

3.3 Configure boot loaders

In grub, make sure have below lines like:

title Gentoo Linux, Xen 4.1 with Kernel 3.1
root (hd0,0)
kernel /boot/xen-4.1.gz
module /boot/kernel-genkernel-x86_64-3.1.6-gentoo root=UUID="whatever" fstype=ext4 ro dolvm
module /boot/initramfs-genkernel-x86_64-3.1.6-gentoo

If you don't know UUID of the rootfs, please run tool blkid.
4 domU configuration

Modify /etc/inittab, add the below lines:

#/etc/inittab
hvc0:2345:respawn:/sbin/getty 38400 hvc0 linux
xvc0:2345:respawn:/sbin/getty 38400 xvc0 linux

Modify /etc/securetty, add below lines:

# hypervisor virtual consoles
hvc0
xvc0

Modify domU grub.conf:

default 0
timeout 5

title Gentoo Linux, Kernel 3.1.6-gentoo
root (hd0)
kernel /boot/kernel-genkernel-x86_64-3.1.6-gentoo root=UUID="whatever" ro dolvm
initrd /boot/initramfs-genkernel-x86_64-3.1.6-gentoo

Please note:

    The domU kernel can be the same kernel as dom0 kernel, because kernel 3.1 can support both dom0 & domU (and KVM host/guest too).
    For domU, there is no need to install grub, we only need a /boot/grub/grub.conf in domU, this file will be parsed by xen-pvgrub.

5 Start domU

In dom0:
5.1 Create domU launcher script:

Say config file test1 includes:

name    = 'xen'

vcpus = 1;

memory  = 512;
maxmem  = 1024;

#pvgrub here
kernel = "/usr/lib/xen/boot/pv-grub-x86_64.gz"
extra = "(hd0)/boot/grub/grub.conf"
#end pvgrub

disk = [ 'phy:/dev/mapper/vg02-debian,xvda1,w' ];

vif = [ 'bridge=xenbr0, mac=fe:39:00:00:01:11' ];

5.2 Start xen services

rc-update add xenstored default

/etc/init.d/xenstored start

/usr/sbin/xenconsoled

5.3 Launch domU

xl create -c test1

If xenconsoled has been launched, we can see ouput immediately. Otherwise we can use xl console to attach. -] will detach current xen console.

Output may like:

# xl create -c test1
Parsing config file test1
xc: error: panic: xc_dom_bzimageloader.c:556: xc_dom_probe_bzimage_kernel: kernel is not a bzImage: Invalid kernel
Booting 'Gentoo Linux, Kernel 3.1.5-gentoo'

root (hd0)
Filesystem type is ext2fs, using whole disk
kernel /boot/kernel-genkernel-x86_64-3.1.5-gentoo root=UUID="aaaaaaaa-bbbb-cccc
-dddd-eeeeeeeeeeee" ro dolvm
initrd /boot/initramfs-genkernel-x86_64-3.1.5-gentoo

close blk: backend=/local/domain/0/backend/vbd/2/51713 node=device/vbd/51713
[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Linux version 3.1.5-gentoo (root@home) (gcc version 4.5.2 (Gentoo 4.5.2 p1.1, pie-0.4.5) ) #1 SMP PREEMPT Tue Dec 27 21:56:33 CST 2011
[    0.000000] Command line: root=UUID="aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee" ro dolvm
[    0.000000] ACPI in unprivileged domain disabled
[    0.000000] released 0 pages of unused memory
[    0.000000] Set 0 page(s) to 1-1 mapping.
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  Xen: 0000000000000000 - 00000000000a0000 (usable)
[    0.000000]  Xen: 00000000000a0000 - 0000000000100000 (reserved)
[    0.000000]  Xen: 0000000000100000 - 0000000020100000 (usable)
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] DMI not present or invalid.
[    0.000000] No AGP bridge found
[    0.000000] last_pfn = 0x20100 max_arch_pfn = 0x400000000
[    0.000000] init_memory_mapping: 0000000000000000-0000000020100000
[    0.000000] RAMDISK: 0200b000 - 027cb000
[    0.000000] No NUMA configuration found
[    0.000000] Faking a node at 0000000000000000-0000000020100000
[    0.000000] Initmem setup node 0 0000000000000000-0000000020100000
[    0.000000]   NODE_DATA [000000001fffb000 - 000000001fffffff]
[    0.000000] Zone PFN ranges:
[    0.000000]   DMA      0x00000010 -> 0x00001000
[    0.000000]   DMA32    0x00001000 -> 0x00100000
[    0.000000]   Normal   empty
[    0.000000] Movable zone start PFN for each node
[    0.000000] early_node_map[2] active PFN ranges
[    0.000000]     0: 0x00000010 -> 0x000000a0
[    0.000000]     0: 0x00000100 -> 0x00020100
[    0.000000] SFI: Simple Firmware Interface v0.81
[    0.000000] SMP: Allowing 1 CPUs, 0 hotplug CPUs
[    0.000000] No local APIC present
[    0.000000] APIC: disable apic facility
...
localhost ~ # cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 23
model name      : Pentium(R) Dual-Core  CPU      E5700  @ 3.00GHz
stepping        : 10
cpu MHz         : 2992.556
cache size      : 2048 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu de tsc msr pae cx8 sep cmov pat clflush mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc up rep_good nopl aperfmperf pni ssse3 cx16 hypervisor lahf_lm dts
bogomips        : 5985.11
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

localhost ~ #

6 Misc

I did a simple test, compile ncurses-5.9 in both dom0 & domU with make -j1, in domU:

real    1m18.058s
user    0m48.501s
sys     0m24.422s

And in dom0:

real    1m18.684s
user    0m41.670s
sys     0m33.490s

Hardware: E5700, DDR3 8G, HDD (500G, 5400RPM).