[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-users] Cannot boot with LVM inside Xen DomU

Hi Oliver,
Thanks a ton for all the valuable information.

I am definitely not using the file based images. I am using LVM based root images for Xen guests using
 disk = [ 'phy:/dev/Box1/debian-lvm-image,xvda,w' ]

One very important point that I get from you is that doing rsync won't work for copying LVM based Dom0. So I am going to try dd command to copy the entire partition to a logical volume (/dev/Box1/debian-lvm-image).

What still I am not getting is that what is issue with nested LVM configuration. Consider this ,

/dev/Box1/debian-lvm-image will exist on Dom0 (node running the Xen hypervisor)
/dev/Boc1/root will exist on DomU

I can't visualize the existence of two volume groups in the system causing a conflict.

>The following is more likely:
I checked the Dom0 grub configuration and observed that root is set to 

This is the reason I was using root=/dev/mapper/Box1-root   (I am sorry I mentioned incorrect root=/dev/Box1/root in my last reply.)
I will try to use root=/dev/xvda and keep the /etc/fstab intact.

I  don't want to avoid installing Xen guest (what are the possible ways of installing a PV guest with LVM inside ? Should I try virt-install? ).

All the info you provided has certainly filled many blanks. I will respond back with the results using the new image.


On Sun, Jul 10, 2011 at 6:59 PM, Oliver Wilcock <oliver@xxxxxxx> wrote:
If you used a file based image for the guest, then your VM config will need to refer to that image.

The following is not a file based image:
> disk = [ 'phy:/dev/Box1/debian-lvm-image,xvda,w' ].

I don't use file based images, but I think you do something like this:
disk = [ 'file:/home/Box1/image.raw,xvda,w' ].

If you are already doing LVM then I don't see the value of file based images.

Where you are really on the wrong track is with the idea that the LVM is copied during rsync of a file system.

You could copy an LVM partitioned disk but you would use a block/bit based tool like dd.

If you did that, though, you would would create a nested LVM configuration.  I think that will cause you grief.  Two volume groups with the same name are going to be in conflict.

The following config lines are in conflict:
> and root='/dev/Box1/root
> Withing the virtual disk image, the /etc/fstab contains the entries
> /dev/xvda       /

The following is more likely:

This tells the guest kernel were to find the root image before it reads the fstab.

The idea of using the Dom0 kernel is possible, but harder to maintain in the long run.  If you are making hundreds of light-weight VMs you might think the extra effort is worth it, but then you wouldn't copy the entire root file system, would you...

So, based on the snippets of your configuration file it appears that you are trying to use a LVM-backed guest image.  Good choice; but poor description.

Mounting the image to copy in the root file system should work.  You picked the same file system, right?  If the Dom0 was using ext3 then the guest should be the same (cause the Dom0 initrd that you are recycling contains the driver for ext3).  Using /dev/xvda will work but I have found, when using SUSE, that this creates challenges when trying to upgrade the guest from one release to another.  The install programs expect a disk with a primary partition (or at least, they did the first time I tried this).  As a consequence, I typically partition the guest image and use /dev/xvda1 for the root file system.

I hope the above fills in some of the the blanks in your understanding.  With Xen you get lots of flexibility, but not all of the choices should necessarily be used together.

To better communicate your challenges it would help if you identified the "error".

So you want an VM inside your guest?  And you want to avoid installing the guest?  You will need to mount the guest image in the Dom0 and create an LVM with in it.

Here is a hint:
xm block-attach 0 phy:/dev/Dom0guest-space/debian-lvm-guest-image xvda w
pvcreate /dev/xvda
vgcreate Box1 /dev/xvda
lvcreate -n box1-boot -L20GB /dev/Box1

On 2011-07-10 04:24, Prakhar Srivastava wrote:
Thanx a lot for the response.
"Populated Xen guest image with Dom0)" - Take a look at
I have used a file based image and then formatted and mounted it at
/mnt. I copied the entire root disk as described using rsync to the file
based virtual disk image. Once I have the virtual disk image which i
believe should contain the LVM partitions (because the Dom0 system was
LVM partitioned), I copy it to a new logical volume
(say /dev/Box1/debian-lvm-image) which then I use to create a xen domU.

I am using the kernel and initrd of th dom0.

disk = [ 'phy:/dev/Box1/debian-lvm-image,xvda,w' ].
and root='/dev/Box1/root

Withing the virtual disk image, the /etc/fstab contains the entries
/dev/xvda       /
/dev/xvdb       swap
(I use a file based image for swap).

The setup described above does gives an error while booting DomU.
Please let me know if I should clarify further. My overall objective is
to run LVM wihin Xen guest VM.


On Sun, Jul 10, 2011 at 5:21 AM, Oliver Wilcock <oliver@xxxxxxx
<mailto:oliver@xxxxxxx>> wrote:

   I have used LVM inside a guest VM.

   I don't understand your description though.
   In particular I don't know what the following means: "Populated Xen
   guest image with DomO image."  You copied the bits from one LV to
   the new LV?

   If you are trying to avoid installing the guest OS by copying the
   bits, or the file system from your Dom0, then you are going to need
   to explain to it were to find root.

   If the LV that you created for your guest is /dev/Box1/root then the
   guest will need to use this name.

   i.e. /dev/Box1/root

   You will need to pass the entire disk into the guest.  Perhaps:

   disk = [ 'phy:/dev/sdb,xvda,w' ]

   To override the fstab and initrd expectations of the name of the
   root file system you can set the kernel parameter in your guest VM
   configuration.  Perhaps:

   Oh, and if the guest is managing the volume group then the Dom0
   can't. So before you mount the physical disk in the DomU you need to
   deactivate it in Dom0.

   vgchange -a n /dev/Box1

   If I've misunderstood and you are trying to create a nested LVM
   configuration I think you are going to run into trouble having two
   volume groups with the same name.

    > volume.
      You copied the bits from the

   On 2011-07-09 15:00, xen-users-request@xxxxxxxxxensource.com
   <mailto:xen-users-request@lists.xensource.com> wrote:

       Date: Sat, 9 Jul 2011 22:27:41 +0530
       From: Prakhar Srivastava<prakhar.apj@xxxxxxx_com
       Subject: [Xen-users] Cannot boot with LVM inside Xen DomU
       Content-Type: text/plain; charset="iso-8859-1"

       I have Xen 4.0 installation on Debian Squeeze. I am able to boot
       Xen guest
       based on physical partition but failing to boot Xen guest with
       LVM within
       DomU. To use LVM inside DomU, I took the following way:
       1. Installed a basic debian squeeze system on a AMD64 box. Used
       the entire
       disk disk and configure LVM option to set root on LVM. The root
       is at
       /dev/Box1/root and swap is at /dev/Box1/swap_1
       2. Installed Xen and Dom0 from repository.
       3. Populated Xen guest image with DomO image.

       When I try to run the image, I get the error that the volume
       logical volume
       is not found. I have set root to /dev/mapper/root in Xen
       Is there anything I am missing. Has anyone used LVM inside Xen


   Xen-users mailing list
   Xen-users@xxxxxxxxxxxxxxxxxxx <mailto:Xen-users@lists.xensource.com>

Xen-users mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.