Re: [Xen-devel] PCI delegation works, access to the delegated NIC doesn't

[Ryan <hap9@xxxxxxxxxxxxxx>]

On Wed, 2006-03-08 at 11:36 +0000, Keir Fraser wrote:
> On 7 Mar 2006, at 20:34, Alexander Wilms wrote:
> 
> >
> > I delegated successfully my NIC to a domU, but access to it from domU 
> > is not
> > possible. Device appears, but access to it via 'ifconfig ethX up' 
> > doesn't
> > work. Ends up in dmesg output like:
> > tg3: tg3_reset_hw timed out for eth1, firmware will not restart 
> > magic=00000000
> >
> > Delegating my USB controllers e.g. works flawlessly.
> 
> Another worthwhile thing to try is the following (in domain0):
> 
> cat /proc/pci
> cat /sys/bus/pci/devices/0000\:02\:00.0/resource
> python /usr/lib/python/xen/util/pci.py 0 2 0 0
> 
> And send us the output (or attach to the bugzilla report and let us 
> know on the list).
> 
>   -- Keir

Based on a quick glance at the source code of the tg3 linux device
driver, it appears that it is one of those PCI devices which doesn't do
what the specification recommends and puts device specific registers in
the PCI configuration space. I guess there's nothing really bad about
that, but it will cause breakage with PCI driver domains in Xen. The PCI
backend intercepts all reads/writes to the PCI configuration space. It
blocks almost all writes and allows almost all reads. I suspect that the
driver is trying to write to the configuration space on the card and is
being denied by the PCI backend.

Alex, try turning on verbose request in the backend and only enable your
ethernet card in the frontend. Do a:

echo 1 > /sys/modules/pciback/parameters/verbose_request

or boot you dom0 with "pciback.verbose_request=1" on the kernel
command-line. Then start up the domain that has the ethernet device.
This will cause a lot of debug output to appear in the kernel log in
dom0. In particular, look for lines that begin with "pciback:
0000:02:00.0: write request " in your logs. Some write requests are ok
(like the one to the PCI_COMMAND register), but most others will be
denied. If you could add this output to your bugzilla entry, that would
be helpful. You can match the register offsets with standard PCI
registers in include/linux/pci_regs.h and you may be able to match the
writes to registers specific to your card in drivers/net/tg3.h

I've suspected that such cards exist (that use device-specific registers
in the configuration space), but have not yet seen one in practice. If
this is indeed the problem, one solution would be to detect the card in
the PCI backend and modify its virtual configuration space to allow
writes to those specific registers (a pciback "quirks" capability
similar to the quirks capability for PCI devices in Linux). This
wouldn't be difficult, but it could get out-of-control if we have to do
that for a lot of cards. Another possibility is to add some kind of flag
that allows all writes to work that are not specifically blocked. I
don't like that idea at all (default permit is not a good security
posture), but would allow people with devices like this one to get them
working (albeit with less protection). Any other suggestions?

Ryan


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel