|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [Xen-users] 9.0-RELEASE PV from scratch on XCP v1.1.0
Apologies for the re-post. Forgot to sign original. -Trx
Title: HOWTO-FreeBSD-on-XCP
Author: John D. "Trix" Farrar
Date: 2012-02-09
* The Challenge - ParaVirtualized FreeBSD 9.0-RELEASE under XCP 1.1.0
The idea here is to create a PV FreeBSD VM under Xen Cloud Platform
(XCP) without starting with an HVM first. The documentation I''ve
been able to find on-line is at least a year old and is mostly written
for Xen (under Linux or NetBSD) rather than XCP. To make things more
interesting, the hardware on which I have XCP installed doesn''t
support Hardware Virtualization -- so I can''t just install an HVM and
convert it over. A PV under XCP means using pygrub as a bootloader to
load the FreeBSD kernel. It seems that pygrub doesn''t know enough
about BSD disk slices and partitions to be able to find the kernel on
its own, so it will need a patch
This solution isn''t perfect, but I think it''s a good start.
* The Equipment
Yes, my hardware is old/cheap.
* Xen Cloud Platform server - XCP v1.1.0
* Build Server - FreeBSD 9.0-RELEASE (32-bit)
* NFS Server - anything w/ enough space. Can be the Build
Server. (should NOT be the XCP Server)
* The Process
* Patching pygrub
Adrian Chadd (http://wiki.freebsd.org/AdrianChadd/XenHackery) did
a great job laying the groundwork for a Xen (not XCP)
installation. I also got a lot of help from Adian''s patch to
pygrub at http://people.freebsd.org/~adrian/xen/bsd_pygrub/. The
problem is that Adrian''s patch "is against the pygrub shipped with
the xen-3.0.3-80.el5_3.2 CentOS 5.3 package." Using Adrian''s
patch, I wsa able to create a patch against the pygrub shipped
with XCP v1.1.0:
==BEGIN==
Patch attached.
===END===
* Creating a disk image
I mounted an exported filesystem from the NFS server so that I
would have a large, sharable space in which to place the disk
image. This space is also mounted on the XCP server. In both
cases, the NFS filesystem is mounted under /mnt.
The following steps are taken on the BUILD host where /usr/src has
already been compiled with buiidworld and buildkernel targets.
==BEGIN==
export FSIMAGE="FreeBSD-PV.img"
truncate -s 10G /mnt/${FSIMAGE} # Create the file where the image
will reside.
mdconfig -a -t vnode -f /mnt/${FSIMAGE} -u 0 # Create the image itself
fdisk -BI md0 # Write the DOS partition table
# Create an inital BSD Label
# For testing, we'll assume one large partition
bsdlabel -w -B md0s1
bsdlabel -e md0s1 # Change partition a from 'unused' to '4.2BSD'
newfs -O1 /dev/md0s1a # UFS, because pygrub doesn't grok UFS2
# Mount up the image and do the installation
mount /dev/md0s1a /media
# Perform the installation to the disk image.
cd /usr/src
make -s DESTDIR=/media KERNCONF=XEN installworld
make -s DESTDIR=/media KERNCONF=XEN installkernel
make -s DESTDIR=/media KERNCONF=XEN distribution
# Set up the ttys(5) file so the VM will use Xen's console.
cat >>/media/etc/ttys <<EOF
# Xen Console
xc0 "/usr/libexec/getty Pc" vt100 on secure
EOF
# Create fstab(5) so the root (only) filesystem will mount.
cat >>/media/etc/fstab <<EOF
# Device Mountpoint FStype Options Dump Pass#
/dev/xbd0s1a / ufs rw 1 1
EOF
# Start and rc.conf(5) file
# NOTE: You won't be able to login via SSH until users are added.
cat >>/media/etc/rc.conf <<EOF
hostname="${VMNAME}"
ifconfig_xn0='dhcp'
sshd_enable='YES'
EOF
# BUILD Server - Unmount the filesystem, the image is done.
umount /media
mdconfig -d -u 0
===END===
The following section is the script I actually used to create a VM
that is more-or-less realistic as a template.
==BEGIN-Advanced-Option==
truncate -s 20G /mnt/${FSIMAGE}
mdconfig -a -t vnode -f /mnt/${FSIMAGE} -u 0
fdisk -BI md0
bsdlabel -w -B md0s1
cat >>xcp-optionb-bsdlabel.txt <<EOF
# /dev/md0s1:
8 partitions:
# size offset fstype [fsize bsize bps/cpg]
a: 512MB 63 4.2BSD 0 0 0
b: 512MB * swap
c: 41929587 0 unused 0 0 # "raw" part, don't edit
d: 512MB * 4.2BSD 0 0 0
e: 2097152 3145791 4.2BSD 0 0 0
f: 6291456 5242943 4.2BSD 0 0 0
g: 12582912 11534399 4.2BSD 0 0 0
h: 17812276 24117311 4.2BSD 0 0 0
EOF
bsdlabel -R md0s1 xcp-optionb-bsdlabel.txt
rm xcp-optionb-bsdlabel.txt
# root partition has to be UFS1. PyGRUB chokes on UFS2 (fsimage.ufs, actually)
newfs -O1 /dev/md0s1a && mount -v /dev/md0s1a /media
newfs /dev/md0s1d && mkdir /media/tmp && mount -v /dev/md0s1d
/media/tmp
newfs /dev/md0s1e && mkdir /media/var && mount -v /dev/md0s1e
/media/var
newfs /dev/md0s1f && mkdir /media/usr && mount -v /dev/md0s1f
/media/usr
newfs /dev/md0s1g && mkdir /media/usr/local && mount -v /dev/md0s1g
/media/usr/local
newfs /dev/md0s1h && mkdir /media/home && mount -v /dev/md0s1h
/media/home
pushd /usr/src
make -s DESTDIR=/media KERNCONF=XEN installworld installkernel distribution
popd
cat >>/media/etc/ttys <<EOF
# Xen Console
xc0 "/usr/libexec/getty Pc" vt100 on secure
EOF
cat >>/media/etc/fstab <<EOF
# Device Mountpoint FStype Options Dump Pass#
/dev/xbd0s1a / ufs rw 1 1
/dev/xbd0s1f /usr ufs rw 1 1
/dev/xbd0s1g /usr/local ufs rw 1 1
/dev/xbd0s1d /tmp ufs rw 1 1
/dev/xbd0s1e /var ufs rw 1 1
/dev/xbd0s1h /home ufs rw 1 1
EOF
cat >>/media/etc/rc.conf <<EOF
hostname="freebsd-9.0"
ifconfig_xn0='dhcp'
sshd_enable='YES'
EOF
umount -v /media/home /media/usr/local /media/usr /media/var /media/tmp /media
===END-Advanced-Option===
* Create the Virtual Host
I got the bulk of this information from Grant McWilliams'' HOWTO
for installing CentOS on an XCP host
(http://grantmcwilliams.com/item/563-centos6-on-xcp).
All of the next steps are to be carried on at the Local Command
Shell prompt on the XCP host.
==BEGIN==
VMNAME="FreeBSD-9.0" # Edit this value, each VM must be different
# XCP Server - Create the Virtual Disk Image and copy .img file into it.
export SR_UUID=$(xe sr-list name-label="Local storage" --minimal)
export VDIUUID=$(xe vdi-create \
sr-uuid=${SR_UUID} \
name-label=${VMNAME}-vdi \
type=system \
virtual-size=11GiB )
# Pull the built image into a VDI (Virtual Disk Image)
# This operatoin will take a while. (like an hour and a half)
xe vdi-import \
uuid=${VDIUUID} \
filename=/mnt/${FSIMAGE}
# Create a VM from the "Other install media" template
# Then customize to our taste
export TPLUUID=$(xe template-list \
name-label="Other install media" \
params=uuid --minimal)
# Create the VM itself
export VM_UUID=$(xe vm-install \
new-name-label=${VMNAME} \
template=${TPLUUID})
# Customize
# Defaults:
# Desc. Value VM Parameter(s)
# RAM = 256 MB = memory-static-max, memory-dynamic-max, memory-dynamic-min
# 128 MB = memory-static-min
# CPUs = 1 = VCPUs-at-startup, VCPUs-max
xe vm-param-set uuid=${VM_UUID} name-description="New FreeBSD host"
xe vm-param-clear uuid=${VM_UUID} param-name=HVM-boot-policy
xe vm-param-clear uuid=${VM_UUID} param-name=HVM-boot-params
xe vm-param-set uuid=${VM_UUID} PV-bootloader=pygrub
xe vm-param-set uuid=${VM_UUID}
PV-bootloader-args="--kernel=/boot/kernel/kernel"
xe vm-param-set uuid=${VM_UUID} other-config:disable_pv_vnc=1
## xe vm-param-set uuid=${VM_UUID} other-config:mac_seed=''
# Create the Network Interface
SWITCH="xenbr0" #
# The vendor prefix 00:16:3e is assigned to Xen in oid.txt
MAC="00:16:3e:12:34:56" # Edit this value, each VIF must be different
## MAC="random" # if you prefer randomly assigned addresses
# Get an ID for the virtual switch.
# It is assumed that this is on the first bridge.
export NETUUID=$(xe network-list bridge=${SWITCH} --minimal)
export VIFUUID=$(xe vif-create \
vm-uuid=${VM_UUID} \
network-uuid=${NETUUID} \
mac=${MAC} \
device=0)
# Create the Virtual Block Device that links the
# VDI to the VM
export VBDUUID=$(xe vbd-create \
vm-uuid=${VM_UUID} \
device=0 \
vdi-uuid=${VDIUUID} \
type=Disk \
mode=RW \
bootable=true )
===END===
* Booting it up
==BEGIN==
# Starting the first time takes a while
xe vm-start uuid=${VM_UUID}
DOMID=$(xe vm-list uuid=${VM_UUID} params=dom-id --minimal)
/usr/lib/xen/bin/xenconsole ${DOMID}
===END===
# The main difference on the XCP side is that the disk image is
bigger. The other steps are the same. I threw in a condenced line to
start the VM and fork the console just because it's easier to paste
into a terminal window.
==BEGIN-Advanced-Option==
VDIUUID=$(xe vdi-create \
sr-uuid=${SR_UUID} \
name-label=${VMNAME}-vdi \
type=system \
virtual-size=21474836480 )
xe vm-start uuid=${VM_UUID} && \
/usr/lib/xen/bin/xenconsole $(xe vm-list uuid=${VM_UUID} params=dom-id
--minimal)
===END-Advanced-Option===
* Open Issues
I'm probably missing something really obvious here, but here are a
couple of issues that keep this from being a complete win.
* Kernel ignoring/not-getting arguments from the domain builder.
This means that the kernel can''t find its root device.
* The pygrub loader won''t execute /boot/loader, so the kernel has to
be called directly. No access to loader.conf(5) and the
customization it affords.
--
John D. "Trix" Farrar __\\|//__ Basement.NET
trix@xxxxxxxxxxxx (` o-o ') http://www.basement.net/
-----------------------------------ooO-(_)-Ooo--------------------------
GPG Key Fprint: 525F DBA7 1A62 E4C4 E642 DF95 384B B851 3CEF C10A
Attachment:
pygrub.xcp-ufs-debug.patch Attachment:
pygrub.xcp-ufs-debug.patch.sig Attachment:
pgpK6UZXKebPD.pgp _______________________________________________ Xen-users mailing list Xen-users@xxxxxxxxxxxxxxxxxxx http://lists.xensource.com/xen-users
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |