Re: [Xen-devel] PCI device export

[Mark Williamson <mark.williamson@xxxxxxxxxxxx>]

> I'm trying to get to work InfiniBand card in DomU via exporting PCI device.

OK, cool.

> How does the export works internally?
>
> I guess that pci frontend forwards stuff like pci_read_config_dword and
> pci_write_config_dword to pci backend that actually does it. Am I right?

IIRC, yes that is what is done.  However the backend has various policies over 
what it allows.  For instance, I think it doesn't by default allow writes to 
BARs or to parts of config space not owned by this device.

> So if the driver of InfiniBand writes some config values to PCI config
> space and these values are not applied actually, then the problem is in pci
> front and back ends. Right? I.e., the PCI backend does not write the values
> to the device.

That sounds right, yes.  I think there's a permissive mode for the backend 
where it lets things through, and there's a means of listing devices that 
require more permissive behaviour on a per-device basis.

> It is currious that PCI frontend returns correct values. 
> Could it be caused by some caching that the frontend may do?

Uhhhh, possibly, I guess.  Maybe it's updating it's own internal data 
structures but the changes aren't getting applied?

Have you checked if error messages are appearing in dmesg in either the domU 
or the dom0?

> I've reported some troubles with InfiniBand cards in Xen but nobody cares.
> Nor the developers of the IB drivers nor people in the xen-dev list. :(

I care!  IB on Xen should be really useful.

There was a project (which I assume is still ongoing...?) to support 
virtualisation of IB devices in Xen, so that you can give several (PV 
initially) domains direct access to an IB device.  This would leverage the IB 
device's safe user accessibility feature to make this passthrough safe for 
untrusted domains whilst eliminating dom0 from the IO path.

AFAIK those changes haven't been applied to the mainline Xen tree.  There's 
another tree where they were being developed, but I don't know a) how well it 
works b) how up-to-date it is or c) whether it's currently being worked on.

Hope that helps you some!

Cheers,
Mark

-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel