HannibalWiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
======Building a Dom0======
Various distributions deliver the base to create a Xen domain-zero (Dom0). Below a few step-by-step CLI guides on getting from bare metal to a Dom0 for PV or HVM.



=====Suse=====
====OpenSuse 11.4====

Install a minimal OpenSuse system, choose "other" graphical system, and choose text based server).

Change the default runlevel to 3, and change the default boot-option to Xen, and use fixed IP-numbers etc.


Install the Xen hypervisor and a Xen-enabled Linux kernel, and the bridging stuff with:
  zypper install  


or use Yast: Virtualization -> Install Hypervisor and Tools -> Xen to install the Xen hypervisor and dom0 patched kernel.



===bridged networking===
Configure in /etc/xen/xend-config.sxp the following network-script line (and comment out all other network-script lines): 
  (network-script hannibal-xen-network-script)

Leave the line as is:
  (vif-script vif-bridge)


And create a file (chmod +x) /etc/xen/scripts/hannibal-xen-network-script (an example for a machine with four physical NIC's:
  #!/bin/sh
  BASEPATH=/etc/xen/scripts
  ${BASEPATH}/network-bridge "$@" vifnum=0 netdev=eth0 bridge=xenbr0
  ${BASEPATH}/network-bridge "$@" vifnum=1 netdev=eth1 bridge=xenbr1
  ${BASEPATH}/network-bridge "$@" vifnum=2 netdev=eth2 bridge=xenbr2
  ${BASEPATH}/network-bridge "$@" vifnum=3 netdev=eth3 bridge=xenbr3


===References===
Further reading:
  * [[http://www.scribd.com/doc/43734889/Xen3-Zero-to-Xen-From-the-CLI-in-OpenSuSE-10-3]]
  * 



=====Debian=====
====Debian 6.0 (Squeeze)====

Install a minimal Debian-6 (Squeeze) system in 64 bit. Preferrably configured with a static IP-configuration in /etc/network/interfaces (ie. DHCP-client not tested).

Prepare the system installing your usual administration tools of choice and some required for our Xen networking. For example:
  apt-get install ssh dnsutils tcpdump bzip2 bridge-utils (etc.)


Install the Xen hypervisor and a Xen-enabled Linux kernel. This package will also install some dependencies as Python and iproute.
  apt-get install  xen-hypervisor-4.0-amd64  linux-image-xen-amd64

To get Xen HVM support
  apt-get install xen-qemu-dm-4.0


===grub2 corrections and tuning===
IMPORTANT! Make the Xen-hypervisor the first boot entry in grub:
  mv /etc/grub.d/20_linux_xen /etc/grub.d/08_linux_xen
  
Append the stanza below to /etc/default/grub to prevent the prober to list al VM's:
  # Disable OS prober to prevent virtual machines on logical volumes from appearing in the boot menu.
  GRUB_DISABLE_OS_PROBER=true

To fix the Dom0 assigned memory to a static value, add to /etc/default/grub
  # Xen4 dom0 and Grub2:
  GRUB_CMDLINE_XEN_DEFAULT="dom0_mem=256M"

Finally, update the grub2 config:
  update-grub2



===bridged networking===
When the physical machine (ie the dom0) has more than 1 physical NIC, configure in /etc/xen/xend-config.sxp the following network-script line (and comment out all other network-script lines): 
  (network-script hannibal-xen-network-script)

and comment out the line:
  #(vif-script vif-bridge)


And create a file /etc/xen/scripts/hannibal-xen-network-script (an example for a machine with four physical NIC's:
  #!/bin/sh
  BASEPATH=/etc/xen/scripts
  ${BASEPATH}/network-bridge "$@" vifnum=0 netdev=eth0 bridge=xenbr0
  ${BASEPATH}/network-bridge "$@" vifnum=1 netdev=eth1 bridge=xenbr1
  ${BASEPATH}/network-bridge "$@" vifnum=2 netdev=eth2 bridge=xenbr2
  ${BASEPATH}/network-bridge "$@" vifnum=3 netdev=eth3 bridge=xenbr3


And make the script executable:
  chmod 755 /etc/xen/scripts/hannibal-xen-network-script



Now there is a working setup, with multiple bridges. However, only the bridge which is configured with an IP-adress with a working gateway is activated fast. It takes a few minutes for the other bridges to come up. This is a bug in /etc/xen/scripts/network-bridge (squeeze/jan 2011)

The fix is to subsititute the line 118 with lines 119-121:

  118         #[ -n "$gateway" ] && ip route add default via ${gateway}
  119         if [ -n "$gateway" ]; then
  120             ip route add default via ${gateway}
  121         fi


See also: [[http://lists.xensource.com/archives/html/xen-users/2010-06/msg00420.html]]


==network devices==
If your hardware has a firewire device, udev will automatically create network devices (i.e. ethN) for them. Great, is'n it :-)  If you do not want this, edit  /etc/udev/rules.d/70-persistent-net.rules  and give them a proper name like firewN or completely disable them.

Add contrib and non-free to /etc/apt/sources and install the Realtec binary-firmware, and update your initrd image:

  apt-get install firmware-realtek
  apt-get install firmware-linux-nonfree
  update-initramfs -u



===Refs===
See also:
  * [[http://wiki.debian.org/Xen]]
  * [[http://wiki.xensource.com/xenwiki/Xen4.0]]



====Debian 5.0 (Lenny)====
See Debian 4.0

====Debian 4.0 (Etch)====
===Core OS and hypervisor===
==create a bootable system==
With the availability of 64-bit hardware in mind, we only use the AMD64 port of Debian Etch in order to create Xen enabled servers. There are several reasons for this decision, one of them is to avoid the TLS-issue's with the 32-bit Xen hypervisor on x86 CPU's.

Configure /etc/apt/sources.list just for the regular Debian repositories.

  deb http://ftp.nl.debian.org/debian/ etch main contrib
  deb-src http://ftp.nl.debian.org/debian/ etch main contrib
  deb http://security.debian.org/ etch/updates main contrib
  deb-src http://security.debian.org/ etch/updates main contrib

Then run:
  apt-get update && apt-get dist-upgrade -fu

Prepare the system installing your usual administration tools of choice. For example:
  apt-get install ssh dnsutils tcpdump bzip2 (etc.)

The following package is required for our Xen networking configuration:
  apt-get install bridge-utils

Install the Xen hypervisor and a Xen-enabled Linux kernel. This package will also install some dependencies as Python and iproute.
  apt-get install xen-linux-system-2.6.xx-x-xen-amd64              (2.6.xx-x is 2.6.18-5 January 2008)
or
  apt-get install xen-linux-system-2.6.xx-x-xen-vserver-amd64      (incl. vserver support)

The required initial ramdisk is automatically rebuilt and the GRUB-config is adjusted for the new hypervisor/kernel combination.
By default we add a Xen boot option to the hypervisor line in /boot/grub/menu.lst so that the hypervisor won't consume all memory. Although Xen automagically reallocates memory to the domU's, we prefer this boot option. Leave the line itself commented as this is required by grub.

  # xenhopt=dom0_mem=256M

Now you can run:

  update-grub

On 32-bit dom0 systems you'll want to install libc6 libraries that are optimized for the Xen hypervisor (to avoid the TLS issues).

  apt-get install libc6-xen 

A reboot will complete the installation.

==Xen config==
In /etc/default/xendomains, the following option can be changed to:
  XENDOMAINS_SAVE=""

==networking with bridges==
Networking is not trivial with Xen. All traffic goes through bridged NIC's (see: http://wiki.xensource.com/xenwiki/XenNetworking). The default configuration works out-of-the-box for systems with one NIC.
Just leave the configuration in /etc/network/interfaces for the desired IP-configuration of the dom0


When the physical machine (ie the dom0) has more than 1 NIC, configure in /etc/xen/xend-config.sxp the following ''network-script'' line (and comment out all other ''network-script'' lines):
  (network-script hannibal-xen-network-script)

And create a file /etc/xen/scripts/hannibal-xen-network-script using this content (an example for a machine with three physical NIC's:
  #!/bin/sh
  BASEPATH=/etc/xen/scripts
  "$BASEPATH/network-bridge" "$@" vifnum=0 netdev=eth0 bridge=xenbr0
  "$BASEPATH/network-bridge" "$@" vifnum=1 netdev=eth1 bridge=xenbr1
  "$BASEPATH/network-bridge" "$@" vifnum=2 netdev=eth2 bridge=xenbr2

And make executable:
   chmod 755 /etc/xen/scripts/hannibal-xen-network-script

(See also: http://www.debian-administration.org/articles/470).


In /etc/network/interfaces the regular eth0 has to be swapped now with for example:

  auto xenbr0
  iface xenbr0 inet static
    address       10.64.100.239
    network       10.64.100.0
    netmask       255.255.255.0
    broadcast     10.64.100.255
   gateway       10.64.100.1

In /etc/rc.local the following has to be added to make the bridges which have no IP-number working for the domU's:

  /sbin/ifconfig xenbr0 up
  /sbin/ifconfig xenbr1 up
  /sbin/ifconfig xenbr2 up


==udev==
If you have troubles with udev creating firewire network devices that mess up your intended network configuration, rename them to a higher ''ethX'' number in:
  /etc/udev/rules.d/z25_persistent-net.rules


==virtual machine administration==
To enable the http-based admin tool, place the following in /etc/xen/xend-config.sxp
  (xend-unix-server yes)
  (xend-port        8000)

===Performance tuning===
Not all linux default kernel parameters are optimal for the dom0 job. Add the part below to /etc/sysctl.conf for some network optimizations:

  # ## some Xen dom0 optimizations:
  net.core.rmem_max=4194304
  net.core.wmem_max=4194304
  net.core.netdev_max_backlog=2048
  net.core.somaxconn=512



===Hypervisor updates from backports===
At this moment backports contains already the 3.2 version of the hypervisor with the credit-schedular. To use this, add the line below to /etc/apt/sources.list
  deb http://www.backports.org/debian etch-backports main contrib non-free

and install with:
  apt-get install xen-hypervisor-3.2-1-amd64 xen-utils-3.2-1 xen-utils-common xen-linux-system-2.6.18-6-xen-amd64

This installs the new hypervisor and utils, but does not make any changes to /boot/grub/menu.list so make these manually.




====Debian 3.0 (Sarge)====
===Core operating system and hypervisor===
==create a bootable system==
Start with the install of a minimal working Debian-sarge on your bare metal. Download a debian-31r2-i386-netinst.iso for this. At the end of the install procedure, answer no to the options task-select and dselect. As the result of this, you end up wit a minimal system. Use apt-get softwarepackagename to install stuff you need. For example ssh could be useful.


Use /dev/sda1 or /dev/hda1 for your root filesystem. This is a Xen requirement! The use of a 64bit is highly recommended. Only when you have 32bits hardware use the plain i386 version.

==repository==
append the following line to /etc/apt/sources.list
  ## ################# Xen-stuff:
  ## (replace <DISTRO> with either sid, sarge or etch and <ARCH> with i386 or amd64.)
  deb     http://packages.debianbase.de/<DISTRO>/<ARCH>/xen3 ./
  deb-src http://packages.debianbase.de/sources/xen3 ./
  #
 

and use the command:
  apt-get update
  apt-get install xen linux-xen0-2.6      (or a newer version, if available :-)
to install the debian-base Xen packages.
Read and follow the instructions mentioned during the install of this package carefully. You have to make an initial-ramdisk for your situation and an addition to menu.list (grub).

and use the command:
  apt-get install bridge-utils
for some extra stuff.

==post aptget==
In case of an i386 architecture do:
  mv /lib/tls /lib/tls.disabled

Install the kernel as a package:
  dpkg -i /usr/src/kernel-xen.version-here-xxxxxxxxxxxxxxxx.deb

Make a initial-ramdisk:
  mkinitrd -o /boot/xen-modules-2.6.12.6-xen 2.6.12.6-xen

Add a section to the grub menu.list:
  title     Xen 3.0.1 - Linux 2.6.12.6-xen
             kernel      /boot/xen-3.0.1.gz dom0_mem=131072
             module      /boot/xen-linux-2.6.12.6-xen root=/dev/sda1 ro
             module      /boot/xen-modules-2.6.12.6-xen

==networking with bridges==
When using more than 1 physical xen host on an ethernet subnet, place a # before the commands in the script: /etc/xen/scripts/network-bridge on the follow lines:
  #       ip link set ${pdev} addr fe:ff:ff:ff:ff:ff
  #       ip link set ${netdev} addr fe:ff:ff:ff:ff:ff


Create an /etc/init.d/xennetworking and place an /etc/rc2.d/S20 symlink
<code bash>
#!/bin/bash
#
# This scripts sets up the Xen networking environment.
# It uses the existing Xen scripts to create bridges on the host machine and to add interfaces to them.
#
# eth0 = realtec 8169       = OOB1
# eth1 = via rhine          = internet
#
# Use Xen-scripts to create bridges
/etc/xen/scripts/network-bridge start bridge=xenbr0 netdev=eth0 vifnum=0
/etc/xen/scripts/network-bridge start bridge=xenbr1 netdev=eth1 vifnum=1
/etc/xen/scripts/network-bridge start bridge=xenbr2 netdev=eth2 vifnum=2
#
# Patience is good
sleep 2
#
####################
# Set IP-configuration on bridges
#
# OOB1 = 10.66.1.121/24
# OOB2 = -
# Internet = nvt.
/sbin/ip addr add 172.17.1.5/24 dev xenbr0
#
# Set default route
/sbin/ip route add default via 172.17.1.1
</code>



=====Redhat=====

====Redhat 6.x ====
RHEL-6 does not ship Xen hypervisor or tools and does not ship Xen dom0 capable kernel.

It can be build, see: http://wiki.xen.org/xenwiki/RHEL6Xen4Tutorial




=====Dom0 additions=====

====Debian====
===Volume manager===
We prefer to run our Xen guests from logical volumes instead of running them from loop mounted files. By default we choose to format the file systems of the Xen guests to xfs.
Install the necessary LVM and XFS packages:

  apt-get install lvm2 dmsetup xfsprogs xfsdump

Create an LVM (8E-type) partition using fdisk and prepare a logical volume for the domU configuration files. We'll mount this volume on /opt/xen.

  pvcreate /dev/sda5
  vgcreate vg00 /dev/sda5
  lvcreate -n optxen -L 20G vg00
  mkdir /opt/xen
  mkfs.xfs /dev/vg00/optxen

Add this mountpoint to /etc/fstab so that the volume will be automagically remounted on system reboots.

  /dev/mapper/vg00-optxen   /opt/xen   xfs   defaults   0   2

Mount the partition we just created.

  mount -a

Now we can create logical volumes for our Xen guests. Don't forget to add a volume per domU for swap!

If you want you might continue reading the section [["xen:creation_of_a_domu"|creation_of_a_domu]].



====Power====
For APC equipment, there is software ready in the Debian repository. Good reason to buy this if you need uninterrupted power. After connecting the equipment (power and USB-cabling), install the ups-daemon software:

  apt-get install  apcupsd apcupsd-doc apcupsd-cgi


Edit the configfile /etc/apcupsd/apcupsd.conf

<code>
## apcupsd.conf v1.1 ##
#
#  for apcupsd release 3.14.8 (16 January 2010) - debian
#
# "apcupsd" POSIX config file
UPSNAME ups01
UPSCABLE usb
UPSTYPE usb
DEVICE
LOCKFILE /var/lock
ONBATTERYDELAY 6
BATTERYLEVEL 90
MINUTES 10
TIMEOUT 0
ANNOY 300
#
UPSCLASS standalone
UPSMODE disable
#
STATFILE /var/log/apcupsd.status
EVENTSFILE /var/log/apcupsd.events
</code>

And flag the parameter ISCONFIGURED in /etc/default/apcupsd to yes.
HannibalWiki

User Tools

Site Tools

Page Tools
