Differences

This shows you the differences between two versions of the page.

--- virtualisation:kvm [2016/02/14 18:28]
Luc Nieland [Debian 6 (Squeeze) or 7 (Wheezy) or 8 (jessie)]
+++ virtualisation:kvm [2020/02/13 17:05] (current)
Luc Nieland CentOS-8: bond + vlan + bridge
@@ Line 3: / Line 3: @@
-=====Debian 6 (Squeeze) or 7 (Wheezy) or 8 (jessie)=====
+=====Distribution: Debian 6 (Squeeze) or 7 (Wheezy) or 8 (jessie)=====
 ===preparation===
 Install a minimal debian system with a static IP-number.
@@ Line 33: / Line 33: @@
-===config===
+===network config===
 Add to /etc/network/interfaces
 <code>
@@ Line 167: / Line 167: @@
-=====Ubuntu 12.04 / 14.04 LTS =====
+=====Distribution: Ubuntu 12.04 / 14.04 LTS =====
 Use the Debian 6 howto.
@@ Line 195: / Line 195: @@
   dpkg-reconfigure locales
+=====Distribution: Ubuntu 18.04 LTS =====
+Install the networking software:
+  apt-get install  bridge-utils vlan
+===network config===
+This uses netplan.
+Remove all .yaml files in /etc/netplan/ (or rename them to *.disabled)
+==One network, one bridge==
+A simple configuration for a simple network. The server has one bridge, with a static IP on the bridge.
+Add to  /etc/netplan/10-netconfig-bridge-static.yaml
+<code>
+network:
+  version: 2
+  renderer: networkd
+  ethernets:
+    eno1:
+      dhcp4: no
+  bridges:
+    br0:
+      interfaces:
+        - eno1
+      addresses:
+        - 192.168.2.203/24
+      gateway4: 192.168.2.1
+      parameters:
+        stp: false
+        forward-delay: 0
+      nameservers:
+        addresses:
+          - 194.109.6.66
+          - 194.109.9.99
+</code>
+==One bridge per VLAN==
+A more complex setup. The system has one physical NIC. This is connected to a switch. This port is in trunc-mode, and has four VLANs configured on this port.
+On the server, the four VLANs are split, and for every VLAN a bridge is created. A static IP number is configured on one bridge, to access the server.
+Remove alle files in /etc/netplan/ and create the file /etc/netplan/10-netconfig-bridge-per-vlan.yaml with the following:
+<code>
+network:
+  version: 2
+  renderer: networkd
+  ethernets:
+    eno1:
+      dhcp4: no
+      dhcp6: no
+  bridges:
+    br0010:
+      interfaces:
+        - vlan0010
+      parameters:
+        stp: false
+        forward-delay: 0
+      addresses:
+        - 192.168.10.42/24
+      gateway4: 192.168.10.1
+      nameservers:
+        addresses:
+          - 1.1.1.1
+          - 8.8.8.8
+    br0011:
+      interfaces:
+        - vlan0011
+      parameters:
+        stp: false
+        forward-delay: 0
+      dhcp4: no
+      dhcp6: no
+    br0012:
+      interfaces:
+        - vlan0012
+      parameters:
+        stp: false
+        forward-delay: 0
+      dhcp4: no
+      dhcp6: no
+    br0013:
+      interfaces:
+        - vlan0013
+      parameters:
+        stp: false
+        forward-delay: 0
+      dhcp4: no
+      dhcp6: no
+  vlans:
+    vlan0010:
+      accept-ra: no
+      id: 10
+      link: eno1
+    vlan0011:
+      accept-ra: no
+      id: 11
+      link: eno1
+    vlan0012:
+      accept-ra: no
+      id: 12
+      link: eno1
+    vlan013:
+      accept-ra: no
+      id: 13
+      link: eno1
+</code>
+And add the following file: /etc/systemd/network/10-netplan-brUp.network
+<code>
+[Match]
+Name=br00*
+[Network]
+LinkLocalAddressing=no
+ConfigureWithoutCarrier=true
+</code>
+Explanation. This is to bring up the anonymous bridges automatically after boot (the bridges which have no ip-address configured on it). Due to a bug in the combination of netplan and networkd, anonymous bridges will have operational status 'off' after boot.
+This can be checked with:
+  networkctl list
+This can be solved manually with:
+  ip link set dev br0011 up
+  ip link set dev br0012 up
+  ip link set dev br0013 up
+for the above netplan yaml example.
+===KVM software===
+Install the KVM-serversoftware:
+  apt-get install  qemu-kvm libvirt-daemon-system virt-top
+And the cli administration tools:
+  apt-get install libvirt-clients
+=====Distribution: CentOS =====
-=====CentOS 6.0 =====
 ===preparation===
-Install a minimal CentOS 6.0 system with a static IP-number.
+Install a minimal CentOS system with a static IP-number.
+===network config CentOS-8===
+With nmcli
+Add to /etc/sysconfig/
+Bonding:
+<code>
+ nmcli con add type bond con-name bond0 ifname bond0  autoconnect yes \
+         ipv4.method disabled  \
+         ipv6.method ignore
+ nmcli con add type ethernet ifname eno1 con-name bond0-sl1    master bond0
+ nmcli con add type ethernet ifname eno2 con-name bond0-sl2    master bond0
+<code>
+Split the trunc-datastream to VLAN's:
+<code>
+ nmcli con add type vlan ifname vlan20 con-name vlan20 vlan.id 20 \
+       vlan.parent bond0 \
+       ipv4.method disabled \
+       ipv6.method ignore
+ # repeat per VLAN
+<code>
+Create a bridge per VLAN:
+<code>
+BR_NAME="br20"
+BR_INT="vlan20"
+SUBNET_IP="192.168.103.32/24"
+GW="192.168.103.1"
+DNS1="192.168.102.144"
+DNS2="192.168.102.146"
+nmcli connection add    type bridge  con-name ${BR_NAME} ifname ${BR_NAME}  autoconnect yes
+ nmcli connection modify ${BR_NAME} ipv4.method manual   ipv4.addresses ${SUBNET_IP}
+ nmcli connection modify ${BR_NAME} ipv4.gateway ${GW}
+ nmcli connection modify ${BR_NAME} ipv4.dns ${DNS1} +ipv4.dns ${DNS2}
+ nmcli connection up ${BR_NAME}
+nmcli connection add type bridge-slave  con-name ${BR_INT} ifname ${BR_INT} master ${BR_NAME}    autoconnect yes
+ nmcli connection up ifname ${BR_INT}
+#
+ip r add default  via 192.168.103.1
+#
+<code>
+===hypervisor kvm===
 Install the software:
   yum install kvm virt-manager libvirt
+=====Check and performance tuning=====
+Do a final check on the host with:
+  virt-host-validate
+<code>
+  QEMU: Checking for hardware virtualization                                 : PASS
+  QEMU: Checking if device /dev/kvm exists                                   : PASS
+  QEMU: Checking if device /dev/kvm is accessible                            : PASS
+  QEMU: Checking if device /dev/vhost-net exists                             : PASS
+  QEMU: Checking if device /dev/net/tun exists                               : PASS
+  QEMU: Checking for cgroup 'memory' controller support                      : PASS
+  QEMU: Checking for cgroup 'memory' controller mount-point                  : PASS
+  QEMU: Checking for cgroup 'cpu' controller support                         : PASS
+  QEMU: Checking for cgroup 'cpu' controller mount-point                     : PASS
+  QEMU: Checking for cgroup 'cpuacct' controller support                     : PASS
+  QEMU: Checking for cgroup 'cpuacct' controller mount-point                 : PASS
+  QEMU: Checking for cgroup 'devices' controller support                     : PASS
+  QEMU: Checking for cgroup 'devices' controller mount-point                 : PASS
+  QEMU: Checking for cgroup 'net_cls' controller support                     : PASS
+  QEMU: Checking for cgroup 'net_cls' controller mount-point                 : PASS
+  QEMU: Checking for cgroup 'blkio' controller support                       : PASS
+  QEMU: Checking for cgroup 'blkio' controller mount-point                   : PASS
+  QEMU: Checking for device assignment IOMMU support                         : PASS
+  QEMU: Checking if IOMMU is enabled by kernel                               : PASS
+   LXC: Checking for Linux >= 2.6.26                                         : PASS
+   LXC: Checking for namespace ipc                                           : PASS
+   LXC: Checking for namespace mnt                                           : PASS
+   LXC: Checking for namespace pid                                           : PASS
+   LXC: Checking for namespace uts                                           : PASS
+   LXC: Checking for namespace net                                           : PASS
+   LXC: Checking for namespace user                                          : PASS
+   LXC: Checking for cgroup 'memory' controller support                      : PASS
+   LXC: Checking for cgroup 'memory' controller mount-point                  : PASS
+   LXC: Checking for cgroup 'cpu' controller support                         : PASS
+   LXC: Checking for cgroup 'cpu' controller mount-point                     : PASS
+   LXC: Checking for cgroup 'cpuacct' controller support                     : PASS
+   LXC: Checking for cgroup 'cpuacct' controller mount-point                 : PASS
+   LXC: Checking for cgroup 'devices' controller support                     : PASS
+   LXC: Checking for cgroup 'devices' controller mount-point                 : PASS
+   LXC: Checking for cgroup 'net_cls' controller support                     : PASS
+   LXC: Checking for cgroup 'net_cls' controller mount-point                 : PASS
+   LXC: Checking for cgroup 'freezer' controller support                     : PASS
+   LXC: Checking for cgroup 'freezer' controller mount-point                 : PASS
+</code>
+====tuning====
+===PCI passthrough===
+Verify that your system has IOMMU support (VT-d):
+  dmesg | grep -e DMAR -e IOMMU
+or for AMD-machines:
+  dmesg | grep AMD-Vi
+If the hardware supports it, pass one of the following commands as a kernel parameter:
+     intel_iommu=on      # Intel only
+     iommu=pt iommu=1    # AMD only
+For example in /etc/default/grub or /etc/sysconfig/grub in the line
+  GRUB_CMDLINE_LINUX_DEFAULT="...."
+=====Nested KVM=====
+(this feature is only for the purpose of testing, not for production)
+A first check:
+  egrep '(vmx|svm)' /proc/cpuinfo
+will give one or more lines when virtual machines can be created on this host.
+Check the CPU-architecture of the physical system with:
+  lscpu
+Check the current status of the host/hypervisor (the physical system) with:
+  cat /sys/module/kvm_intel/parameters/nested
+To activate KVM nesting, create or edit /etc/modprobe.d/kvm.conf (or /etc/modprobe.d/qemu-system-x86.conf on ubuntu) on the host and add:
+<code>
+options kvm_intel nested=1
+#options kvm_amd nested=1
+</code>
+Reboot the system to effect the setting.
+Create a VM, and use "copy host CPU configuration" in the cpu-section of the VM-definition.
+In this VM you can check the kvm nesting feature with:
+  cat /sys/module/kvm_intel/parameters/nested
+Also:
+  egrep '(vmx|svm)' /proc/cpuinfo
+will give one or more lines.
@@ Line 239: / Line 548: @@
 In case the (v)disk is larger dan 2TB, use parted instead of fdisk.
 Below an example of a 3 TB disk, which is enlarged to 3.5TB. The KVM guest is rebooted, and the filesystem on it is umounted.
-The trick is to delete te partition (rm) and to recreate it directly afterwards, with the same startingpoint, and with a greater endpoint.
+The trick is to delete te partition (rm) and to recreate it directly afterwards, with the same startingpoint, and with a lager number as the endpoint.
+  parted -l /dev/vdc
 <code>
@@ Line 251: / Line 564: @@
 Number  Start   End     Size    File system  Name  Flags
       1049kB  3299GB  3299GB  xfs
+</code>
-(parted) rm 1
+<code>
+# parted /dev/vdc
+(parted) rm 1
+</code>
+<code>
+# parted /dev/vdc
 (parted) mkpart
 Partition name?  []?
 File system type?  [ext2]? xfs
 Start? 1049kB
 End? 3848GB
+</code>
+Instead of a number as an answer to the question "End", it is also possible to use -1. This will use the largest possible value, ie. the end of the disk.
+The action above can also be done in a one-liner:
+<code>
+# parted /dev/vdc
+(parted) mkpart  primary xfs 1049kB -1
+</code>
+View the result:
+<code>
+# parted /dev/vdc
 (parted) p
 Model: Virtio Block Device (virtblk)
@@ Line 267: / Line 602: @@
       1049kB  3848GB  3848GB  xfs
-(parted)
+(parted) q
 </code>
-Now, resize the filesystem on the partition with resizefs or xfs_growfs.
+The partitioning part is done now.
+Resize the filesystem on the partition. For example with resizefs or xfs_growfs.
@@ Line 340: / Line 678: @@
 Now run pvresize on this partition, and after this, the volume-group has free-extents. Use these to grow one or more logical-volumes.
+===add a lun to a VPS===
+To attach a storage lun to a running VPS , use the following procedure:
+On the VM-host:
+Create a new raw-device (and optionally make a symlink).
+Attach the raw-device to the vps (in this case, the name is vpsfoo2 and it is the second virtio-disk aka vdb):
+  virsh attach-disk vpsfoo2 /var/lib/libvirt/images/vpsfoo2data.img vdb --targetbus virtio  --live
+======Migration of VMs to another host ======
+Instructions how to migrate VMs to another hypervisor-host.
+====Offline====
+Create a destination KVM-hypervisor system, including bridges on the required networks and VLANs. Try to use the same names for bridges, filesystems, logical-volumes. Else use "virsh edit" to make the modifications befor starting the VM on the destination hypervisor.
+===On the source-hypervisor===
+create a definition-file:
+  virsh list --all
+  virsh dumpxml --security-info  vpstest2  >  /var/lib/libvirt/images/vpstest2.xml
+  virsh shutdown vpstest2
+  virsh destroy  vpstest2   # if needed
+===On the destination-hypervisor===
+Create the required logical-volumes, and symlinks:
+  lvcreate -L 4G -n vpstest2 vg0
+  ln -s /dev/mapper/vg0-vpstest2 /var/lib/libvirt/images/vpstest2.img
+And get the raw-logical-volume with a dd piped through ssh:
+  ssh root@sourcehyp "dd if=/dev/mapper/vg0-vpstest2" | dd of=/dev/mapper/vg0-vpstest2
+And get the config-definition-file:
+  scp root@sourcehyp:/var/lib/libvirt/images/vpstest2.xml  /var/lib/libvirt/images/vpstest2.xml
+And create the VM:
+  virsh define /var/lib/libvirt/images/vpstest2.xml
+And start the VM:
+  virsh start vpstest2

HannibalWiki

User Tools

Site Tools

Differences

Page Tools