FreeBSD Xen HVM DomU

These are my notes on setting up a FreeBSD Xen HVM DomU

PLEASE NOTE: This content contains outdated and possibly irrelevant information.

You might draw some insight from this information and that is awesome, but please be aware that this content does not constitute a comprehensive guide and is more just a set of notes for my own reference.

LVM Storage

You can use LVM volumes for disk partitions in Xen environments when you do not have better alternatives.

Assuming available space on your volume group, create some logical volumes:

lvcreate -L 20G -n example-main vg1 && \
lvcreate -L 200G -n example-data vg1 && \
lvcreate -L 8G  -n example-swap vg1

This group of volumes represents a 20GB volume for the root partition, a 200GB partition for user data, and an 8GB volume for swap space.

Loop Device for ISO

These notes apply to the Xen 3.X series, which can use an ISO image as a boot device ala CDROM. If you run into issues, you can configure a loop device on the host machine to enable boot from CDROM with an ISO image.

If you have problems specifying the ISO image directly for CDROM usage, then you can mount a loop device on the Xen Dom0:

losetup -f /root/FreeBSD-8.2-RELEASE-amd64-disc1.iso && \
losetup /dev/loop0

Once you’ve created a loop device, update the disk entries in your Xen configuration to include an entry for the “cdrom drive”:

phy:/dev/loop0,ioemu:hdd:cdrom,r

A complete example Xen DomU configuration for your reference:

# Example FreeBSD 8.2 Xen DomU

name = "example"
kernel = "/usr/lib/xen/boot/hvmloader"
builder = 'hvm'

memory = 2048
shadow_memory = 8

cpus = "1"
vcpus = "0"

vif = [ 'mac=00:23:3e:55:73:78, ip=10.10.1.76, vifname=examplewan, bridge=xenbr0',
        'mac=00:23:3e:77:73:78, ip=192.168.1.76, vifname=examplelan, bridge=xenbr1' ]

disk = [
    # CDROM loop device
    'phy:/dev/loop0,ioemu:hdd:cdrom,r',
    'phy:/dev/vg1/example-main,ioemu:hda,w',
    'phy:/dev/vg1/example-swap,ioemu:hdb,w',
    'phy:/dev/vg1/example-data,ioemu:hdb,w']

boot= 'dc' # cdrom then disk
# boot='cd' # disk then cdrom

serial = 'pty'

# VNC console for installation only
sdl=0
vnc=1
vnclisten='127.0.0.1'
vncconsole=1
vncpassword=''
stdvga=1

on_poweroff = 'destroy'
on_reboot = 'restart'
on_crash = 'restart'

You should change certain values as appropriate to match your own system configuration before trying to use the example.

If you check out the configuration details, you’ll notice some VNC specific directives in the example Xen DomU configuration above. This helps with initial console connections (for example, to install FreeBSD) as they must happen through the VNC frame buffer until the system detects an actual virtual console device.

You can typically do this by enabling the appropriate configuration settings and then using an SSH tunnel to allow for remote VNC to localhost.

An example ssh invocation looks like this:

ssh -l username -L 5900:localhost:5900 remote_hostname

You’ll want to do the following basic steps to make this kind of connection work out:

1. Establish a proper SSH tunnel as in the above example.
2. Start the Xen DomU.
3. Use VNC on your local machine to connect to localhost.

Once you’ve established a console, you can complete the installation.

You can remove the loop0 entry from your configuration when you complete your installation, and change the boot order (boot= in the Xen configuration) to cd to boot from the hard disk first.

A commented version of the sequence appears in the example DomU configuration.

FreeBSD now ships with a standard Xen HVM kernel configuration (XENHVM) that among other things, will build paravirtual drivers, which will increase network and disk performance in the DomU instance.

You can enable the Xen xm console to work properly from this kernel with some configuration changes.

You should build a fully Xen HVM aware custom kernel to take full advantage of your particular hardware environment, desired performance characteristics, capabilities, and so on.

A complete example AMD64 XENHVM kernel configuration for your reference:

ident   POTRZEBIE
machine amd64
cpu     HAMMER
options VESA
options SC_PIXEL_MODE
options VGA_WIDTH90
options SC_DISABLE_REBOOT
options ATA_STATIC_ID
options SMP
options KDB_TRACE
options KDB
options INCLUDE_CONFIG_FILE
options FLOWTABLE
options MAC
options AUDIT
options HWPMC_HOOKS
options KBD_INSTALL_CDEV
options PRINTF_BUFR_SIZE=128
options _KPOSIX_PRIORITY_SCHEDULING
options P1003_1B_SEMAPHORES
options SYSVSEM
options SYSVMSG
options SYSVSHM
options STACK
options KTRACE
options SCSI_DELAY=5000
options COMPAT_FREEBSD7
options COMPAT_FREEBSD6
options COMPAT_FREEBSD5
options COMPAT_FREEBSD4
options COMPAT_FREEBSD32
options COMPAT_43TTY
options GEOM_LABEL
options GEOM_PART_GPT
options PSEUDOFS
options PROCFS
options CD9660
options MD_ROOT
options UFS_GJOURNAL
options UFS_DIRHASH
options UFS_ACL
options SOFTUPDATES
options FFS
options SCTP
options INET6
options INET
options PREEMPTION
options SCHED_ULE
options XENHVM
options NO_ADAPTIVE_RWLOCKS
options NO_ADAPTIVE_MUTEXES
options GEOM_PART_MBR
options GEOM_PART_EBR_COMPAT
options GEOM_PART_EBR
options GEOM_PART_BSD
device  isa
device  mem
device  io
device  uart_ns8250
device  xenpci
device  cpufreq
device  acpi
device  pci
device  ata
device  atadisk
device  ataraid
device  atapicd
device  atapifd
device  atapist
device  scbus
device  da
device  atkbdc
device  atkbd
device  psm
device  kbdmux
device  vga
device  splash
device  sc
device  agp
device  uart
device  miibus
device  re
device  loop
device  random
device  ether
device  vlan
device  tun
device  pty
device  md
device  gif
device  faith
device  firmware
device  bpf

To get the console working, edit /boot/loader.conf, and add the following lines:

boot_multicons="YES"
boot_serial="YES"
comconsole_speed=115200
console="comconsole,vidconsole"

Edit /etc/ttys and activate ttyu0:

ttyu0	"/usr/libexec/getty std.115200"	dialup	on secure

Restart and you’ll see output (and login terminals) via both xm console and vncviewer.

Now you can complete installation and configuration of the new DomU as you’d like, such as public network interface configuration, enabling SSH access, and so on.