Re: [Xen-devel] [PATCH] libxl: added libxl compatibility with physical backend file for NetBSD

[Ian Campbell <Ian.Campbell@xxxxxxxxxxxxx>]

On Fri, 2011-07-22 at 18:07 +0100, Roger Pau Monné wrote:
> 2011/7/22 Ian Campbell <Ian.Campbell@xxxxxxxxxx>:
> > On Fri, 2011-07-22 at 18:36 +0100, Roger Pau Monne wrote:
> I don't like doing it this way either. How are Linux hotplug scripts
> called? I've been looking arround, and there seems to be a set of udev
> rules that call the scripts based on changes regarding devfs, the
> problem is that NetBSD doesn't have devfs, and we have to watch the
> xenstore for changes in the status of the devices to trigger the
> correct scripts.

The hotplugs scripts are called from a udev rule but the event is
triggered (asynchronously) by the backend driver based upon the changing
state node, so ultimately they are triggered from the same xenstore
watch.

It's certainly possible that the sequencing is different between Linux
and NetBSD or that in the Linux case there is some interlock (deliberate
or otherwise) between the kernel side and the hotplug script side which
saves our bacon.

Who is responsible for driving the backend xenstore state machine under
NetBSD? It doesn't look like xenbackendd (which just reacts to state
changes).

>  I also see that Linux hotplug scripts also remove
> entries from xenstore after detaching (xen-hotplug-cleanup), isn't it
> wrong if the entries are deleted in libxl?

It may be redundant but I don't think it is wrong, it should be harmless
for both to remove the backend but if it isn't then I think it should be
the toolstack's responsibility rather than the hotplug script.

> The basic problem with this (if I got it right) is synchronization,
> entries should be deleted after the corresponding hotplug scripts are
> done. This happens sometimes in NetBSD, but it is not guaranted, since
> there's no rendezvous point to tell libxl that all is fine and
> xenstore can be cleaned. I think it would be good to set some kind of
> indication, that hotplug scripts have finished execution before libxl
> deletes the entries in xenstore, and possibly set a timeout, so libxl
> is not waiting forever if something goes wrong.
> 
> Also with xend hotplug scripts where in charge of removing entries
> from xenstore, but in libxl Linux has no support for block-hotplug
> using the loop device (only 'phy' which doesn't involve any read from
> xenstore to unplug), that's why I think this problem has not appeared
> before.

Ah, that may well be it. I had thought that vnd was all in kernel but
now I see that it is setup and torn down in the hotplug script, much
like for loopback on Linux with xend.

On Linux we were able to make the simplifying assumption of not handling
this for phy devices because we had blktap to fall back on. NetBSD
doesn't currently have that option.

I think part of the problem is that the vbd lifecycle and the backing
device's lifecycle (vnd in this case, but could also be an iscsi login
for example) are somewhat entwined via the hotplug scripts etc when
really they should be layered and somewhat independent.

Ian Jackson has the best handle on this stuff and I know he has plans to
move the block script callout stuff forward to better support various
more complex block device types, integrate better with the toolstack and
generally rationalise the existing interfaces etc so I'd like to hear
what he thinks before I suggesting any particular avenue of attack to
this problem. He's travelling right now but may be online intermittently
next week (we're both going to be at the Debian conference in Banja
Luka), I think he's back properly the week after that though.

> > The libxl device teardown stuff is pretty convoluted and I'm reasonably
> > sure it is wrong in several respects, I've been meaning to untangle it,
> > perhaps I should get to that sooner rather than later and maybe fix this
> > issue as a side effect.
> >
> 
> That would be great, I would like to give a hand if I can.

Thanks.

I'm travelling next week and on vacation the week after. But I need to
get onto this soon so it is done for Xen 4.2, since it will an API
incompatible change to libxl and we are hoping to declare the API stable
at that point.

Have you considered coming to the Munich Xen.org hackathon in September?
http://wiki.xen.org/xenwiki/MUC_2011_hackathon. That would be a great
opportunity to work through some of these issues and ensure that the
future direction of libxl's handling of this stuff works for NetBSD as
well as Linux.

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel