Re: [Xen-devel] PCI passthrough with stubdomain

[Marek Marczykowski-Górecki <marmarek@xxxxxxxxxxxxxxxxxxxxxx>]

On Fri, Sep 23, 2016 at 03:27:07PM +0200, Samuel Thibault wrote:
> Marek Marczykowski-Górecki, on Fri 23 Sep 2016 10:48:14 +0200, wrote:
> > I'm still trying to get PCI passthrough working with stubdomain and
> > without qemu in dom0 (even for just vfb/vkbd backends). How is this
> > supposed to work?
> 
> Just as I recall from my memory:
> 
> > 1. Should xen-pcifront in the target domain be used (and consequently,
> > should xen-pciback be set for it)?
> 
> I guess that could work.

Could, or should? ;)

In the meantime, I've found this in xen-pcifront driver:

        static int __init pcifront_init(void)
        {
                if (!xen_pv_domain() || xen_initial_domain())
                        return -ENODEV;

So, it looks like pcifront is not used in PV domain.

> > Currently xen-pciback is set for both
> > stubdomain and target domain, which presents a race condition and
> > xen-pciback refuses to setup one of them.
> 
> Yes, that's expected, for the reason you say.

In the meantime I've tried to modify libxl to not setup pciback for
target domain. This, along with other modifications (see below) improved
the situation.

> * to summarize
> **************
>
> If running PV drivers in the guest, you set the pciback on the guest, be
> it run with stubdom or not. 
> If running plain drivers in the guest,
>   * when not using a stubdom you don't need to set a pciback.
>   * when using a stubdom you need to set a pciback on the stubdom.

Thanks for the explanation!

> So the unfortunate thing is that when using stubdom, you'd have to set
> the pciback either on the guest (to run a PV driver in it), or on the
> stubdom (to run a plain driver in the guest, and let mini-os use PV to
> let qemu pass the board through)

I've tried only on Linux HVM guest and as noted above xen-pcifront
doesn't look to be involved. Is it any different on other OSes?

As for getting PCI passthrough working with stubdomain, for now I hit
those problems:
1. getdomaininfo not allowed from stubdomain (already fixed in master),
2. double setup of pciback (explained above) - for now I've disabled one
of them,
3. race condition between pcifront in mini-os and qemu.

Regarding the last one:
Toolstack during (stub)domain startup setup two things, mostly
asynchronously:
1. pciback/pcifront (through standard xenstore entries)
2. instruct qemu to attach device (by writing "pci-ins" to
device-model/xxx/command)

The later fails if pcifront is not initialized yet. For now I've moved
the second step to be really after the first one, including
synchronization through libxl__wait_for_backend call. 
Is it ok? Or maybe it should be done on stubdomain side (handling
"pci-ins" should wait on pcifront)? I guess the same will apply to
qemu-xen case, or any other stubdomain, so probably better have it in
toolstack, right?

Work-in-progress patches attached. The result is that lspci inside the
guest finally list the device. No idea if it's really working yet.

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

Attachment: 0001-libxl-attach-xen-pciback-only-to-stubdomain.patch
Description: Text document

Attachment: 0002-libxl-attach-PCI-device-to-qemu-only-after-setting-p.patch
Description: Text document

Attachment: signature.asc
Description: PGP signature

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel