Re: [Xen-devel] [PATCH 5/6] xen-pt: Hide MSI-X from xen stubdoms

[Jason Andryuk <jandryuk@xxxxxxxxx>]

On Tue, Mar 12, 2019 at 10:13 AM Roger Pau Monné <roger.pau@xxxxxxxxxx> wrote:
>
> On Tue, Mar 12, 2019 at 09:58:56AM -0400, Jason Andryuk wrote:
> > On Tue, Mar 12, 2019 at 8:38 AM Marek Marczykowski-Górecki
> > <marmarek@xxxxxxxxxxxxxxxxxxxxxx> wrote:
> > >
> > > On Tue, Mar 12, 2019 at 01:04:19PM +0100, Roger Pau Monné wrote:
> > > > On Mon, Mar 11, 2019 at 02:02:15PM -0400, Jason Andryuk wrote:
> > > > > MSI-X is not supported in Xen stubdoms, so it must be disabled.  Use 
> > > > > the
> > > > > existing xen_pt_hide_dev_cap to hide when running under -xen-stubdom.
> > > >
> > > > I'm afraid this requires some more context. What's the actual issue
> > > > that prevents MSI-X from working?
> > >
> > > At least missing "Fix PCI passthrough for HVM with stubdomain" series,
> > > but that's mostly on Xen side (+ one change how QEMU enable MSI-X in
> > > config space).
> > > Some of it can be worked around by enabling permissive mode. Jason, did
> > > you had a chance to test it with any MSI-X device?
> > > I'm not aware of anything thing particular that breaks MSI-X but not
> > > MSI. Besides much less devices lying around to test MSI-X...
> >
> > OpenXT and Qubes have used a compile time patch that disabled MSI-X
> > for a long time.  The OpenXT patch description doesn't help:
> > """
> > Currently we do not support MSI-X setup for PCI devices passed through.
> >
> > Although the specification mentions that PCI-e devices might implement only
> > MSI-X there is not a lot of those and mostly none that we have encountered 
> > yet.
> > Considering that, we force devices to use MSI by hiding the MSI-X 
> > capability.
> > """
> >
> > To be honest, I didn't question the reasoning and just made the
> > compile-time disabling into a runtime disabling.
> >
> > I tested with a NEC uPD720200 XHCI controller supporting MSI-X.  There
> > was an error related to setting up MSI-X when I failed to pass the
> > "-xen-stubdom" flag.  I can pull that log when I get back to the
> > machine.  With this patch, MSI-X was hidden in the guest, but dom0
> > showed MSI-X present but unused.
>
> Given that Marek is working on a series to make both MSI and MSI-X
> work for pci-passthrough and stubdomains I'm not sure how did you even
> manage to get plain MSI working. Is there something I'm missing?

Marek's series adds a hypercall to enable MSI/MSI-X since they are not
allowed by default in PCI passthrough.  PCI passthrough also has a
permissive mode to allow access to a device's entire PCI configuration
space including enabling MSI.

Qubes 4.0 has an out-of-tree patch that whitelists enabling MSI in
non-permissive mode - I've tested these patches on Qubes 4.0 with the
MSI-X XHCI device where MSI is enabled but not MSI-X.  Other NICs with
only MSI also work.

OpenXT linux stubdoms use permissive mode PCI passthrough, so the
stubdom can program the PCI config space to enable MSI.  I've tested
the patches there, but none of my OpenXT test systems have MSI-X.  MSI
devices work properly.

Marek also tested a "vanilla" linux stubdom version in his osstest
suite, but that doesn't test passthrough.

If Marek's pending patch series or "permissive" mode will enable
MSI/MSI-X, then maybe this patch should just be dropped in favor of
those options.  I'll test to verify whether MSI-X works with
permissive mode.

Regards,
Jason

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel