[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-users] Xen & DRBD



block-drbd doesn't really work. Well, it WORKS, but that's only half the equation.

There are multiple things that try to access the DomU disk device/ files before the block/hotplug scripts are called. So far I know of:

"xm create": (if you specify a bootloader, it tries to verify the block device is accessible, fails because there's no valid block device in drbd:resource0. It'll try to open "resource0" and fail)

pygrub: it has to read the root domU device; if this is a drbd resource it gets passed the resource name and not the /dev/drbdX and it fails. Even if it somehow got passed /dev/drbdX, it wouldn't be Primary and the open+read will fail.


qemu-dm: only relevant if you're doing HVM. in that case your domU will just not see the disk and fail in various ways.

In short, the only way I think it even has a hope of working without lots of wrapping scripts and possibly patching some of the xen python libraries is to do a para-virtualized domU, and specify the kernel/ initrd/etc in the domU config file. (In other words don't use pygrub)

I've actually gotten it working by writing wrappers around qemu-dm, pygrub and adding a bit of code to the xm command. It's all very ugly and not really the way to solve the problem. I think I'm just going to have linux-ha change the disk state when needed, although I have to verify migration will still work properly then.

-=Zak

On Mar 18, 2008, at 4:20 AM, Stuart Poulton wrote:

Hi,

I've just tried again, this time using one of the example xen files.
Details below.


x-host-3:/etc/xen/vm # cat test
# This is a bootloader used to boot paravirtualized domains. You can optionally # plug in a different boot loader here, e.g., pygrub. There is usually no # reason to change this. Don't explicitly specify kernel and ramdisk if you use
# a bootloader.
bootloader="/usr/lib/xen/boot/domUloader.py"

# The domUloader will pull the specified kernel and initrd out of the domU's # disk, and use that to boot. This is easier to manage, compared to manually # keeping a copy of the kernel and initrd in sync in dom0's filesystem and # pointing to them with the "kernel" and "ramdisk" parameters. Syntax is # "VDEV:KERNEL,INITRD", where VDEV is the block device (from domU's point of
# view) within which KERNEL and INITRD can be found.
bootentry = "hda1:/boot/vmlinuz-xen,/boot/initrd-xen"


# Remaining settings for the example VM:
name="domUloader-example"
memory=512
vcpus=1
on_crash="destroy"
on_poweroff="destroy"
on_reboot="restart"
localtime=0
builder="linux"
bootargs="--entry=xvda1:/boot/vmlinuz-xen,/boot/initrd-xen"
disk=[ 'drbd:xendomains,xvda,w' ]
vif=[ 'mac=00:16:3e:00:01:02,bridge=xenbr0' ]
vfb=['type=vnc,vncunused=1']


x-host-3:/etc/xen/vm # cat /proc/drbd
version: 8.0.6 (api:86/proto:86)
SVN Revision: 3048 build by phil@mescal, 2007-09-03 10:39:27
0: cs:Connected st:Primary/Primary ds:UpToDate/UpToDate C r---
   ns:0 nr:1052672 dw:1052672 dr:0 al:0 bm:284 lo:0 pe:0 ua:0 ap:0
       resync: used:0/31 hits:65650 misses:142 starving:0 dirty:0
changed:142
act_log: used:0/257 hits:0 misses:0 starving:0 dirty:0 changed:0
x-host-3:/etc/xen/vm # drbdadm state xendomains
Primary/Primary


x-host-3:/etc/xen/vm # xm create -c test
Using config file "./test".
Error: Disk isn't accessible


x-host-3:/etc/xen/vm # xm create -n opensuse-1
Using config file "./opensuse-1".
(vm
   (name opensuse-1)
   (memory 384)
   (on_poweroff destroy)
   (on_reboot restart)
   (on_crash destroy)
   (vcpus 1)
   (on_xend_start ignore)
   (on_xend_stop ignore)
   (uuid 7bf59311-0304-d34e-09a7-4d2f2fa5f5e9)
   (bootloader /usr/lib/xen/boot/domUloader.py)
(bootloader_args '--entry=xvda1:/boot/vmlinuz-xen,/boot/initrd- xen')
   (image (linux (args ' ')))
   (device (vbd (uname drbd:drdb0) (dev xvda) (mode w)))
   (device (vbd (uname phy:/dev/cdrom) (dev xvdb:cdrom) (mode r)))
   (device (vif (mac 00:16:3e:19:69:48)))
   (device (vkbd))
   (device
       (vfb
           (vncunused 1)
           (type vnc)
           (display localhost:10.0)
           (xauthority /root/.Xauthority)
       )
   )
)



On Mon, 2008-03-17 at 19:28 +0000, Gareth Bult wrote:
Hi,

When you switch to "drbd:" you need to change the path to the drbd resource name, rather than the device name. So if you have a drbd.conf that says;

resource fred {
  on nodea {
      device /dev/drbd0;

You would use;
disk=[ 'drbd0:fred,xvda,w',]

hth
Gareth.


----- Original Message -----
From: "Stuart Poulton" <swp@xxxxxxxxxxxxxx>
To: xen-users@xxxxxxxxxxxxxxxxxxx
Sent: Monday, March 17, 2008 3:08:47 PM GMT +00:00 GMT Britain, Ireland, Portugal
Subject: [Xen-users] Xen & DRBD

Dear All,

I'm using drbd to provide a HA pair of servers to run xen domains on.

I've created a test domain as follows

x-host-2:~ # more /etc/xen/vm/opensuse
name="opensuse"
ostype="opensuse"
uuid="ee62226a-8487-cc02-d89f-dafafd1f35e6"
memory=1214
vcpus=2
on_crash="destroy"
on_poweroff="destroy"
on_reboot="restart"
localtime=0
builder="linux"
bootloader="/usr/lib/xen/boot/domUloader.py"
bootargs="--entry=xvda1:/boot/vmlinuz-xen,/boot/initrd-xen"
extra=" "
disk=[ 'phy:/dev/drbd0,xvda,w',]
vif=[ 'mac=00:16:3e:29:fb:80', ]
vfb=['type=vnc,vncunused=1']

This works fine, and start and stop the domain on both of the nodes.
However if I try to use

disk=[ 'drbd:xendomains,xvda,w',]

I get the following error....

x-host-2:~ # xm create opensuse
Using config file "/etc/xen/vm/opensuse".
Error: Disk isn't accessible

I'm expecting the block-drbd script to do something here, but from what
I can tell it's not.

I suspect I've missed something fundamental. Any pointers please ?

Thanks

Stuart


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.