In the day job, I’m in the process of migrating the VM estate from an aging collection of Xen hosts to Proxmox.
All of the VM’s are CentOS (v5 to v7, with a couple of v4!) with a mixture of bare partitions and LVM-based installs.
TBH, the last time I had to deal with Xen professionally was back when you had to compile it yourself as a kernel module.
Comparatively, Proxmox is awesome. Not just because you can do live migrations without shared storage…😉
This should help https://ycnrg.org/xen-to-kvm-node-migration/
Firstly, check if the VM is using a partition or the raw disk, as if raw disk then grub has nowhere to write to…
Before Migration – common
sudo yum install kernel -y
sudo sed -i -e s/^HWADDR/#HWADDR/1 /etc/sysconfig/network-scripts/ifcfg-eth0
sudo sed -i -e s/^HWADDR/#HWADDR/1 /etc/sysconfig/network-scripts/ifcfg-eth1
# Only centos 5/6 have inittab
sudo sed -i -e s/^co/#co/1 /etc/inittab
CHECK THAT THE NETWORKING DOESN’T HAVE SOME WEIRD SETUP (staging.creativestudioportal.maglabs.net had config for 2 interfaces bonded on the wrong ip!)
cat /etc/sysconfig/network-scripts/ifcfg-eth0
also check for default gateway being on primary interface
NOTE: Some CentOS 5 machines aren’t finding the centosplus kernel packages.
sudo rpm -i kernel-2.6.18-238.19.1.el5.centos.plus.x86_64.rpm
(adjust the above for your desired CentO)S version as the vault site has all the packages)
if bare partition
sudo vim /boot/grub/menu.lst # ← centos 5/6
grub.conf changes aren’t needed on centos7 as we’ll be using grub2 and having that generate its own config file
remove “console=hvc0 xencons=tty” from the entry for the non-xen kernel now installed (eg 2.6.32-504.30.3.el6.centos.plus.x86_64)
Here is where you can also change the default boot entry to the non-xen kernel if you don’t want to do it later…
If lvm partitions (or both?)
cat /etc/sysconfig/networking/profiles/default/resolv.conf
if this contains nameservers you’re not expecting:
nameserver 1.2.3.4
nameserver 2.2.3.4
# replace nameserver addresses as appropriate
sudo -s echo “nameserver 4.4.4.4” > /etc/sysconfig/networking/profiles/default/resolv.conf
# later: whatever needed here to get the serial console working and not breaking!?
# check lvm size on source machine
lvdisplay /dev/HostVolGroup00/ipam.example.net
or
lvs # for the full list
# create destination VM in proxmox using cores/ram/disk info from spreadsheet – with disk a bit bigger (+1G) if moving raw partition
# NOTE: If the guest VM uses LVM, select IDE as the disk controller or LVM will not initialise!
# NOTES: CentOS 5 requires network to be an e1000 otherwise the networking will freeze remote ssh sessions and require rebooting.
# CentOS 7 Guest seems to require disk to be IDE either way
# don’t “start after created” as we want to overwrite the created lvm slice with the source from the xen host
# find the destination LV (here 109 is the VM ID just created)
ls /dev/mapper/* | grep vm-109
/dev/pve/vm–109–disk–0
# if no partition, do this
fdisk /dev/pve/vm-109-disk-0 # (set boot flag with option ‘a’)
type: n p a w q
partprobe /dev/pve/vm-109-disk-0
kpartx -av /dev/pve/vm-109-disk-0
ls /dev/mapper/pve-vm–109–disk–0p1 # <- use this as destination for dd
To make this easier later (you can’t paste into the novnc terminal!), before powering off the source vm, copy & paste these.
To run to mount & make chroot
#!/bin/bash -x
# do-chroot.sh 20210427 smeacher – grabbing from somewhere else is easier?
mount –bind /dev /mnt/sysimage/dev
mount –bind /sys /mnt/sysimage/sys
mount –bind /proc /mnt/sysimage/proc
chroot /mnt/sysimage
#!/bin/bash -x
# do-grub.sh 20210427 smeacher – because the EOF thing doesn’t like quotes in sed
DRIVE=$(lsblk | awk ‘/^[sv]da / {print $1}’ )
MAJOR=$(cat /etc/redhat-release | tr -dc ‘0-9.’|cut -d \. -f1)
case $MAJOR in
7)
grub2-install –force /dev/$DRIVE # from here because it’s weird and a bit shit
grub2-mkconfig -o /boot/grub2/grub.cfg # builds the ‘new grub config’
;;
*)
# means 5 or 6
sed -i “s#xvda#$DRIVE#g” /boot/grub/device.map
grub-install /dev/$DRIVE
;;
esac
If you have a spare machine with a webserver running, you can make the above easier by hosting them there:
sudo curl -o /do-chroot.sh http://mywebserver.net/do-chroot.sh ;
sudo curl -o /do-grub.sh http://mywevserver.net/do-grub.sh ;
sudo chmod +x /do-chroot.sh /do-grub.sh
OPTIONAL: record the uptime😉 (some of those I’ve been moving were up for 5+ years!)
POWEROFF THE SOURCE VM
If bare partition, on destination host:
nc -l -p 19000 | dd bs=16M of=/dev/mapper/pve-vm–109–disk–0p1
If lvm partitions, on destination host:
# start a netcat receiver on the destination
nc -l -p 19000 | dd bs=16M of=/dev/pve/vm-109-disk-0
# run the netcat sender on the source machine
dd bs=16M if=/dev/HostVolGroup00/ipam.example.net | pv | nc 10.24.37.0 19000
Wait for the filesystem to copy… (about 150G/hr on gigabit)
If bare partition
# boot from CentOS 7 iso to install grub & change network settings to suit (if required & sed above for ifcfg-eth0 not done)
#start as rescue system, and as long as it mounts the ‘found’ system:
/mnt/sysimage/do-chroot.sh
cd /
./do-grub.sh
# if the partition isn’t found (CentOS 7 I’m looking at you!)
mount /dev/sda1 /mnt/sysimage
/mnt/sysimage/do-chroot.sh
cd /
./do-grub.sh
NOTE: update /etc/fstab if required – if the drive/partition name has changed (eg /dev/sda to /dev/xda)
If lvm partitions
So far these mostly ‘just work’.
if you’ve not done this already:
boot the VMselect standard kernel at grub prompt, then make that the default in /boot/grub/menu.lst once it has started.
reboot to make sure it works.
That’s basically it.
Once all done, disable the source VM from auto-starting by removing the symlink from /etc/xen/auto
IMPORTANT: don’t forget to
kpartx -d /dev/mapper/pve-vm–109–disk–0p1 # or equivalent
& unmount ISO from proxmox gui
otherwise, they will not migrate properly because of the attached virtual cdrom (have had to do the same in VMWare)