Re: [Xen-devel] xen_emul_unplug on xen 4.1, HVM guest 2.6.38

[Ian Campbell <Ian.Campbell@xxxxxxxxxx>]

On Wed, 2011-10-26 at 18:43 +0100, Alex Bligh wrote:
> Ian,
> 
> > I'm a bit fuzzy on the details but I'm not sure what this has to do with
> > the host, the device naming and behaviour on unplug are kernel side
> > things, I'd expect that if /dev/sdaX as /boot worked on 3.3 it'll work
> > on 4.1 too. (I believe you that it doesn't work, I'm just wondering
> > aloud what I'm missing).
> >
> > Can you give us the specifics of a setup which fails, e.g. a complete
> > guest cfg file, the kernel version, command line options, /etc/fstab,
> > dmesg etc.
> 
> I am not avoiding answering your question (I will get you this) but
> what is /meant/ to happen in the following scenario:

Lets call this scenario A:

> * Install on recent kernel (e.g. 2.6.37) running on Xen 3.3
> * No fancy boot options, xen_emul_unplug not set
> * No XEN_IOPORT_MAGIC implemented, so check_platform_magic()
>   returns an error

Correct, specifically it returns XEN_PLATFORM_ERR_MAGIC.

> * Therefore xen_platform_pci_unplug=0

Correct. There is some special casing around XEN_PLATFORM_ERR_MAGIC but
lets assume xen_emul_unplug=unnecessary has not been passed to the
kernel so it does not take effect.

> * Therefore blkfront etc. don't init (probe returns
>   -ENODEV)
> * Therefore OS boots with root=/dev/sda

sda must be an emulated IDE device since the 2.6.37 kernel does not
support PV devices with names other than xvd*.

I think this is as expected.

> Now Xen 3.3 is upgraded to Xen 4

Lets call this scenario B:

> * Kernel boots, and XEN_IOPORT_MAGIC now exists
> * Therefore unplug occurs, and xen_platform_pci_unplug is non zero
> * Therefore blkfront etc. inits, and PV drivers start
> * OS still boots with root=/dev/sda
> 
> Are the PV devices meant to appear as /dev/sdX rather than /dev/xvdX?

No, there is no support for this in upstream kernels (in general all the
old behaviours of Xen kernels where they would hijack other drivers
device names are not upstreamble)

> If so, how in the code does this happen? If not, won't the boot fail?

You need to be using root=/dev/xvda here for it to work. Or even better
use root=UUID=thing or root=LABEL=thong (many distros do one of these by
default these days).

(I'd forgotten about the UUID=/LABEL= option til just now -- that might
be the bit of magic which was missing to make this work)

> > Hmm, yes I think the special treatment of XEN_IOPORT_MAGIC mismatch on
> > the kernel side is what I was missing.
> >
> > It might make sense to have a guest level config option which disables
> > these magic ports, i.e. makes them return 0xffff like they would have
> > done in 3.3 (I think 0xffff is what you'll get from an invalid port in
> > general).
> 
> Actually I don't think this will work. If we do this,
> xen_plaftofm_pci_unplug will still be zero (as it's only set on exit
> of the function after a successful unplug), and that's enough to
> prevent blkfront and xenbus_probe_frontend from doing anything useful,
> so will effectively disable PV drivers even where they should be enabled.

Correct, this will take you back to scenario A, however if that is how
the guest is configured (to use emulated devices) then this is what you
wanted (or at least it is what the guest configuration is expecting).

If the guest were configured to use PV devices then it would be using
root=/dev/xvda. Such a configuration would have needed
xen_emul_unplug=unnecessary in order to have worked on 3.3 before the
upgrade and this option would be harmless but unnecessary on 4.1.

If you were using UUID=/LABEL= then I think things would have worked in
both cases (emulated on 3.3 and pv on 4.1) without additional kernel
parameters.

One detail worth mentioning is that if the guest is using PV drivers and
expecting xvd* named devices then prior to the unplug the devices
xvd[a-d] are also exposed as the IDE devices hd[a-d]. This is how the
bootloader is still able to access the emulated device.

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel