Re: [Xen-devel] Patching error while setting up COLO

[Ian Campbell <ian.campbell@xxxxxxxxxx>]

On Thu, 2016-02-25 at 16:49 +0800, Changlong Xie wrote:
> Sorry for this. Since we have no privilege to update the Wiki for a long 
> time so most contents of this page are dated : (

Anyone can be granted access, it is only a manual process because of
spammers. Please create an account and then fill in the form at:
http://xenproject.org/component/content/article/100-misc/145-request-to-be-made-a-wiki-editor.html

> 
> Anyway i'll write down detail steps here for colo on XEN.
> 
> [Requirements]
> - Hardware
> There is at least one directly connected nic to forward the network 
> requests from primary to secondary vm. The directly connected nic must 
> not be used by any other purpose. If your guest has more than one nic, 
> you should have directly connected nic for each guest nic. If you don't 
> have enouth directly connected nic, you can use vlan.
> 
> - Dom0
> 1. Kernel with dom0 support
> 2. kernel module
>     nf_conntrack
>     nf_conntrack_ipv4
>     nf_nat
>     libnl-tools >= 3.0.
> *Note*: If your host os has OEM-released xen tools, *MUST* uninstall it 
> first.
> 
> - Guest
> Only HVM guest(without pv extensions) is supported now. If you want to 
> use OEM released guest os, please use SUSE（we use "SUSE Linux 
> Enterprise Server 11" currently）. REDHAT and Ubuntu is not supported 
> now because I don't find any way to disable pv extensions. If you want 
> to use REDHAT or Ubuntu, you need to build the newest kernel which has 
> the parameter xen_nopv.
> 
> 
> [SETUP]
> - Network link topology
> ref: http://wiki.xenproject.org/wiki/COLO_-_Coarse_Grain_Lock_Stepping
> 
> -  Test environment prepare
> 
> On both Primary/Secondary hosts:
> 1.
> cd ~
> git clone https://github.com/wencongyang/colo-proxy
> git clone https://github.com/macrosheep/iptables.git
> git clone https://github.com/torvalds/linux
> git clone https://github.com/wencongyang/qemu-colo
> git clone https://github.com/wencongyang/xen
> 
> 2. Prepare host kernel for Dom0
> colo-proxy kernel module need cooperate with linux kernel. You should 
> patch kernel with ~/colo-proxy/colo-patch-for-kernel.patch
> 1) cd ~/colo-proxy/; git checkout 405527cbfa9f
> 2) cd ~/linux/; git checkout v4.0; git am 
> ~/colo-proxy/colo-patch-for-kernel.patch
> 3) cp /boot/config-3.0.76-0.11-xen  .config; make menuconfig to config 
> your kernel support Dom0.
> Ref: http://wiki.xenproject.org/wiki/Mainline_Linux_Kernel_Configs
> 4) make -j8; make modules_install; make install
> 5) reboot
> 
> 3. build colo-proxy
> 1) cd ~/colo-proxy/; git checkout 405527cbfa9f; make; make install
> 
> 4. build iptables
> 1) cd iptables; ./autogen.sh; ./configure --prefix=/usr/ 
> --libdir=/usr/lib64; make; make install
> 
> 5. build qemu-colo
> 1) cd ~/qemu-colo/; git checkout colo-xen-v2
> 2)
> path_to_xen_source=~/xen
> ./configure --enable-xen --target-list=x86_64-softmmu \
>             --extra-cflags="-I$path_to_xen_source/tools/include 
> -I$path_to_xen_source/tools/libxc -I$path_to_xen_source/tools/xenstore" \
>             --extra-ldflags="-L$path_to_xen_source/tools/libxc 
> -L$path_to_xen_source/tools/xenstore"
> make -j4
> 
> 6. build xen
> 1) cd ~/xen/; git checkout colo_v10
> 2)
> ./autogen.sh
> ./configure --enable-debug
> touch tools/libxl/libxlu_disk_l.l
> touch tools/libxl/libxlu_cfg_l.l
> make dist-xen -j8
> make dist-tools -j8
> make install-xen -j8
> make install-tools -j8
> 
> 7. configure /boot/grub/menu.list to support xen, then reboot
> 
> [ Run COLO ]
> 
> 1.
> on primary:
> rm -f /var/log/xen/*
> rm -f /var/lib/xen/userdata-d.*
> service xencommons start
> modprobe nf_conntrack_ipv4
> modprobe xt_PMYCOLO sec_dev=eth1
> 
> on secondary:
> rm -f /var/log/xen/*
> rm -f /var/lib/xen/userdata-d.*
> service xencommons start
> modprobe xt_SECCOLO
> active_disk=/mnt/ramfs/active_disk.img
> hidden_disk=/mnt/ramfs/hidden_disk.img
> local_img=/root/xie/suse-64hvm.img
> tmp_disk_size=`./qemu-colo/qemu-img info $local_img |grep 'virtual size' 
> > awk  '{print $3}'`
> rm -rf /mnt/ramfs/*
> umount /mnt/ramfs/
> rm -rf /mnt/ramfs/
> mkdir /mnt/ramfs
> function create_image()
> {
>      /root/xie/xen/tools/qemu-xen-dir/qemu-img create -f qcow2 $1 
> $tmp_disk_size
> }
> function prepare_temp_images()
> {
>      grep -q "^none /mnt/ramfs ramfs" /proc/mounts
>      if [[ $? -ne 0 ]]; then
>          mount -t ramfs none /mnt/ramfs/ -o size=2G
>      fi
> 
>      if [[ ! -e $active_disk ]]; then
>          create_image $active_disk
>      fi
> 
>      if [[ ! -e $hidden_disk ]]; then
>          create_image $hidden_disk
>      fi
> }
> prepare_temp_images
> 
> 2. on primary
> xl cr hvm_nopv_colo
> sleep 10
> xl pause hvm_nopv_colo
> xl remus -c -u hvm_nopv_colo 192.168.3.2
> 
> hvm_nopv_colo attached.
> 
> 
> That's all, good luck ; )
> 
> Thanks
>      -Xie
> 
> > Victor
> > 
> > 
> > 
> > 
> > 
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@xxxxxxxxxxxxx
> > http://lists.xen.org/xen-devel
> > 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxx
> http://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel