Xen project Mailing List

[Xen-users] 9.0-RELEASE PV from scratch on XCP v1.1.0

To: FreeBSD Xen <freebsd-xen@xxxxxxxxxxx>, Xen-Users <xen-users@xxxxxxxxxxxxxxxxxxx>

Date: Sat, 11 Feb 2012 10:55:44 -0600

Delivery-date: Mon, 13 Feb 2012 10:20:49 +0000

List-id: Xen user discussion <xen-users.lists.xensource.com>

Title: HOWTO-FreeBSD-on-XCP Author: John D. "Trix" Farrar Date: 2012-02-09 * The Challenge - ParaVirtualized FreeBSD 9.0-RELEASE under XCP 1.1.0 The idea here is to create a PV FreeBSD VM under Xen Cloud Platform (XCP) without starting with an HVM first. The documentation I''ve been able to find on-line is at least a year old and is mostly written for Xen (under Linux or NetBSD) rather than XCP. To make things more interesting, the hardware on which I have XCP installed doesn''t support Hardware Virtualization -- so I can''t just install an HVM and convert it over. A PV under XCP means using pygrub as a bootloader to load the FreeBSD kernel. It seems that pygrub doesn''t know enough about BSD disk slices and partitions to be able to find the kernel on its own, so it will need a patch This solution isn''t perfect, but I think it''s a good start. * The Equipment Yes, my hardware is old/cheap. * Xen Cloud Platform server - XCP v1.1.0 * Build Server - FreeBSD 9.0-RELEASE (32-bit) * NFS Server - anything w/ enough space. Can be the Build Server. (should NOT be the XCP Server) * The Process * Patching pygrub Adrian Chadd (http://wiki.freebsd.org/AdrianChadd/XenHackery) did a great job laying the groundwork for a Xen (not XCP) installation. I also got a lot of help from Adian''s patch to pygrub at http://people.freebsd.org/~adrian/xen/bsd_pygrub/. The problem is that Adrian''s patch "is against the pygrub shipped with the xen-3.0.3-80.el5_3.2 CentOS 5.3 package." Using Adrian''s patch, I wsa able to create a patch against the pygrub shipped with XCP v1.1.0: ==BEGIN== Patch attached. ===END=== * Creating a disk image I mounted an exported filesystem from the NFS server so that I would have a large, sharable space in which to place the disk image. This space is also mounted on the XCP server. In both cases, the NFS filesystem is mounted under /mnt. The following steps are taken on the BUILD host where /usr/src has already been compiled with buiidworld and buildkernel targets. ==BEGIN== export FSIMAGE="FreeBSD-PV.img" truncate -s 10G /mnt/${FSIMAGE} # Create the file where the image will reside. mdconfig -a -t vnode -f /mnt/${FSIMAGE} -u 0 # Create the image itself fdisk -BI md0 # Write the DOS partition table # Create an inital BSD Label # For testing, we'll assume one large partition bsdlabel -w -B md0s1 bsdlabel -e md0s1 # Change partition a from 'unused' to '4.2BSD' newfs -O1 /dev/md0s1a # UFS, because pygrub doesn't grok UFS2 # Mount up the image and do the installation mount /dev/md0s1a /media # Perform the installation to the disk image. cd /usr/src make -s DESTDIR=/media KERNCONF=XEN installworld make -s DESTDIR=/media KERNCONF=XEN installkernel make -s DESTDIR=/media KERNCONF=XEN distribution # Set up the ttys(5) file so the VM will use Xen's console. cat >>/media/etc/ttys <<EOF # Xen Console xc0 "/usr/libexec/getty Pc" vt100 on secure EOF # Create fstab(5) so the root (only) filesystem will mount. cat >>/media/etc/fstab <<EOF # Device Mountpoint FStype Options Dump Pass# /dev/xbd0s1a / ufs rw 1 1 EOF # Start and rc.conf(5) file # NOTE: You won't be able to login via SSH until users are added. cat >>/media/etc/rc.conf <<EOF hostname="${VMNAME}" ifconfig_xn0='dhcp' sshd_enable='YES' EOF # BUILD Server - Unmount the filesystem, the image is done. umount /media mdconfig -d -u 0 ===END=== The following section is the script I actually used to create a VM that is more-or-less realistic as a template. ==BEGIN-Advanced-Option== truncate -s 20G /mnt/${FSIMAGE} mdconfig -a -t vnode -f /mnt/${FSIMAGE} -u 0 fdisk -BI md0 bsdlabel -w -B md0s1 cat >>xcp-optionb-bsdlabel.txt <<EOF # /dev/md0s1: 8 partitions: # size offset fstype [fsize bsize bps/cpg] a: 512MB 63 4.2BSD 0 0 0 b: 512MB * swap c: 41929587 0 unused 0 0 # "raw" part, don't edit d: 512MB * 4.2BSD 0 0 0 e: 2097152 3145791 4.2BSD 0 0 0 f: 6291456 5242943 4.2BSD 0 0 0 g: 12582912 11534399 4.2BSD 0 0 0 h: 17812276 24117311 4.2BSD 0 0 0 EOF bsdlabel -R md0s1 xcp-optionb-bsdlabel.txt rm xcp-optionb-bsdlabel.txt # root partition has to be UFS1. PyGRUB chokes on UFS2 (fsimage.ufs, actually) newfs -O1 /dev/md0s1a && mount -v /dev/md0s1a /media newfs /dev/md0s1d && mkdir /media/tmp && mount -v /dev/md0s1d /media/tmp newfs /dev/md0s1e && mkdir /media/var && mount -v /dev/md0s1e /media/var newfs /dev/md0s1f && mkdir /media/usr && mount -v /dev/md0s1f /media/usr newfs /dev/md0s1g && mkdir /media/usr/local && mount -v /dev/md0s1g /media/usr/local newfs /dev/md0s1h && mkdir /media/home && mount -v /dev/md0s1h /media/home pushd /usr/src make -s DESTDIR=/media KERNCONF=XEN installworld installkernel distribution popd cat >>/media/etc/ttys <<EOF # Xen Console xc0 "/usr/libexec/getty Pc" vt100 on secure EOF cat >>/media/etc/fstab <<EOF # Device Mountpoint FStype Options Dump Pass# /dev/xbd0s1a / ufs rw 1 1 /dev/xbd0s1f /usr ufs rw 1 1 /dev/xbd0s1g /usr/local ufs rw 1 1 /dev/xbd0s1d /tmp ufs rw 1 1 /dev/xbd0s1e /var ufs rw 1 1 /dev/xbd0s1h /home ufs rw 1 1 EOF cat >>/media/etc/rc.conf <<EOF hostname="freebsd-9.0" ifconfig_xn0='dhcp' sshd_enable='YES' EOF umount -v /media/home /media/usr/local /media/usr /media/var /media/tmp /media ===END-Advanced-Option=== * Create the Virtual Host I got the bulk of this information from Grant McWilliams'' HOWTO for installing CentOS on an XCP host (http://grantmcwilliams.com/item/563-centos6-on-xcp). All of the next steps are to be carried on at the Local Command Shell prompt on the XCP host. ==BEGIN== VMNAME="FreeBSD-9.0" # Edit this value, each VM must be different # XCP Server - Create the Virtual Disk Image and copy .img file into it. export SR_UUID=$(xe sr-list name-label="Local storage" --minimal) export VDIUUID=$(xe vdi-create \ sr-uuid=${SR_UUID} \ name-label=${VMNAME}-vdi \ type=system \ virtual-size=11GiB ) # Pull the built image into a VDI (Virtual Disk Image) # This operatoin will take a while. (like an hour and a half) xe vdi-import \ uuid=${VDIUUID} \ filename=/mnt/${FSIMAGE} # Create a VM from the "Other install media" template # Then customize to our taste export TPLUUID=$(xe template-list \ name-label="Other install media" \ params=uuid --minimal) # Create the VM itself export VM_UUID=$(xe vm-install \ new-name-label=${VMNAME} \ template=${TPLUUID}) # Customize # Defaults: # Desc. Value VM Parameter(s) # RAM = 256 MB = memory-static-max, memory-dynamic-max, memory-dynamic-min # 128 MB = memory-static-min # CPUs = 1 = VCPUs-at-startup, VCPUs-max xe vm-param-set uuid=${VM_UUID} name-description="New FreeBSD host" xe vm-param-clear uuid=${VM_UUID} param-name=HVM-boot-policy xe vm-param-clear uuid=${VM_UUID} param-name=HVM-boot-params xe vm-param-set uuid=${VM_UUID} PV-bootloader=pygrub xe vm-param-set uuid=${VM_UUID} PV-bootloader-args="--kernel=/boot/kernel/kernel" xe vm-param-set uuid=${VM_UUID} other-config:disable_pv_vnc=1 ## xe vm-param-set uuid=${VM_UUID} other-config:mac_seed='' # Create the Network Interface SWITCH="xenbr0" # # The vendor prefix 00:16:3e is assigned to Xen in oid.txt MAC="00:16:3e:12:34:56" # Edit this value, each VIF must be different ## MAC="random" # if you prefer randomly assigned addresses # Get an ID for the virtual switch. # It is assumed that this is on the first bridge. export NETUUID=$(xe network-list bridge=${SWITCH} --minimal) export VIFUUID=$(xe vif-create \ vm-uuid=${VM_UUID} \ network-uuid=${NETUUID} \ mac=${MAC} \ device=0) # Create the Virtual Block Device that links the # VDI to the VM export VBDUUID=$(xe vbd-create \ vm-uuid=${VM_UUID} \ device=0 \ vdi-uuid=${VDIUUID} \ type=Disk \ mode=RW \ bootable=true ) ===END=== * Booting it up ==BEGIN== # Starting the first time takes a while xe vm-start uuid=${VM_UUID} DOMID=$(xe vm-list uuid=${VM_UUID} params=dom-id --minimal) /usr/lib/xen/bin/xenconsole ${DOMID} ===END=== # The main difference on the XCP side is that the disk image is bigger. The other steps are the same. I threw in a condenced line to start the VM and fork the console just because it's easier to paste into a terminal window. ==BEGIN-Advanced-Option== VDIUUID=$(xe vdi-create \ sr-uuid=${SR_UUID} \ name-label=${VMNAME}-vdi \ type=system \ virtual-size=21474836480 ) xe vm-start uuid=${VM_UUID} && \ /usr/lib/xen/bin/xenconsole $(xe vm-list uuid=${VM_UUID} params=dom-id --minimal) ===END-Advanced-Option=== * Open Issues I'm probably missing something really obvious here, but here are a couple of issues that keep this from being a complete win. * Kernel ignoring/not-getting arguments from the domain builder. This means that the kernel can''t find its root device. * The pygrub loader won''t execute /boot/loader, so the kernel has to be called directly. No access to loader.conf(5) and the customization it affords. -- John D. "Trix" Farrar __\\|//__ Basement.NET trix@xxxxxxxxxxxx (` o-o ') http://www.basement.net/ -----------------------------------ooO-(_)-Ooo-------------------------- GPG Key Fprint: 525F DBA7 1A62 E4C4 E642 DF95 384B B851 3CEF C10A

_______________________________________________ Xen-users mailing list Xen-users@xxxxxxxxxxxxxxxxxxx http://lists.xensource.com/xen-users

©2013 Xen Project, A Linux Foundation Collaborative Project. All Rights Reserved.
Linux Foundation is a registered trademark of The Linux Foundation.
Xen Project is a trademark of The Linux Foundation.