[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] Hotplugged devices in Xen 4.5 and domain reboot

On 12/1/15 11:05 AM, Ian Campbell wrote:
> On Tue, 2015-12-01 at 18:48 +0200, Iurii Mykhalskyi wrote:
>> Thanks to all for a replays, please see my answers below:
>> On 12/01/2015 05:29 PM, Wei Liu wrote:
>>> On Tue, Dec 01, 2015 at 04:58:55PM +0200, Iurii Mykhalskyi wrote:
>>>> Our real usb mass-storage device are located at driver domain (DomD).
>>>> So we
>>>> setup second block-device backend there.
>>>> To hotplug usb mass-storage from DomD we use follow command:
>>>> xl block-attach domU_id phy:/bla-bla,xvda10,w,backend="DomD"
>>> What happens if you run this in Dom0? I guess DomD doesn't respond to
>>> the request?
>>     Yes, there is no responded from domD, because actual storage device
>> are located there, and toolstack stuck on real device existence check.
> This check is a toolstack bug which should be fixed. We've squashed some of
> them at various points, but I'd not be surprised if others remain.
>>>> There was no support of attaching block-device in runtime from domain
>>>> other
>>>> to Domain-0, so we have made some hacks to allow call block-attach
>>>> command
>>>> from non-dom0 privileged domain.
>>> So this is a special use case. This is the first time I know people
>>> actually run xl block-attach in driver domain.
>>     Yes, this is special case and we this by our solution design.
>>>> One of patches was - don't update
>>>> /var/lib/xen/userdata-d.$DOMID-$UUID.libxl-json during execution of
>>>> this
>>>> command (because this log located on dom0 rootfs and we don't have
>>>> any
>>>> access to it from DomD). So, there is no different in configs before
>>>> and
>>>> after hotplug.
>>> The state of $DOMID is recorded in libxl-json file. No wonder you lose
>>> all state.
>>> But even if you write those states, they are going to be inside driver
>>> domain.  There is no way at the moment to synthesise the state inside
>>> Dom0 and DomD into one. There is also difficulty in how you can split
>>> the synthesised and dispatch the states to multiple entities again when
>>> rebuilding a domain.
>>> So I think having multiple entities managing state of one single domain
>>> is bad. I think the proper way of making it work is to make hotplug
>>> device from domain other than Dom0 work.
>>> There is a daemon "xl devd" in driver domain. We might be able to teach
>>> it to response to Dom0 toostack request. I'm a bit surprised if it
>>> doesn't do that already. Did you forget to start that daemon?
>>     We can't run devd in driver domain, because it failed on connect to
>> xenstored socket (/var/run/xenstored/socket - we have xenstored running
>> only in dom0).   
> devd _should_ be able to talk to xenstored over the kernel provided
> interface to the shared ring rather than the local socket.
> It is certainly not expected that devd be colocated in the same domain as
> happens to be running xenstored.
> If this is not working then there is another bug.

This works, but might have problems in Xen 4.5. If you're using running
on Linux 3.14 or newer then you will have a problem. You need to
backport commit 9c89dc95201ffed5fead17b35754bf9440fdbdc0 if you're using
the C based xenstore and 7d418eab3b6dbdeec84bf73af301dca54368547a if
you're using the Ocaml based xenstore.

You can test this locally by supplying "--disable-socket" to xenstore
when it starts up.

>>> In general a driver domain would not be expected to have sufficient
>>> privilege over e.g. a guest domain's /local/domain/domU/devices to
>>> create
>>> the f.e. dirs.
>>     In our solution we have to create 2 full privileged domains - Dom0 and 
>> DomD, so we need 2 toolstack domains.
>>    Any special privileges hack wasn't done - we need just to setup 
>> additional permissions for DomD.
> I'm afraid you simply cannot have 2 toolstack domains. The toolstack is a
> singleton entity in a Xen system.
> If you want to run toolstack operations from a non-toolstack domain then
> you will need to arrange for some (likely out-of-band) mechanism for such
> domains to ask the single toolstack domain to do something on their behalf.
> Ian.
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxx
> http://lists.xen.org/xen-devel

Doug Goldstein

Attachment: signature.asc
Description: OpenPGP digital signature

Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.