Re: [Xen-devel] Problems accessing passthrough PCI device

[Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>]

On Wed, Nov 19, 2014 at 12:12:09PM -0300, Simon Martin wrote:
> Hello Jan and Konrad,
> 
> Tuesday, November 18, 2014, 1:49:13 PM, you wrote:
> 
> >>
> >> I've just checked this with lspci. I see that the IO is being enabled.
> 
> > Memory you mean.
> 
> Yes. Sorry.
> 
> >> Any   other   idea   on   why I might be reading back 0xff for all PCI
> >> memory area reads? The lspci output follows.
> 
> > Since this isn't behind a bridge - no, not really. Did you try this with
> > any other device for comparison purposes?
> 
> This   is  getting  more  interesting.  It  seems  that  something  is
> overwriting the pci-back configuration data.
> 
> Starting  from a fresh reboot I checked the Dom0 pci configuration and
> got this:
> 
> root@smartin-xen:~# lspci -s 00:19.0 -x
> 00:19.0 Ethernet controller: Intel Corporation Device 1559 (rev 04)
> 00: 86 80 59 15 00 00 10 00 04 00 00 02 00 00 00 00
> 10: 00 00 d0 f7 00 c0 d3 f7 81 f0 00 00 00 00 00 00
> 20: 00 00 00 00 00 00 00 00 00 00 00 00 86 80 54 20
> 30: 00 00 00 00 c8 00 00 00 00 00 00 00 05 01 00 00
> 
> I then start/stop my DomU and checked the Dom0 pci configuration again
> and got this:
> 
> root@smartin-xen:~# lspci -s 00:19.0 -x
> 00:19.0 Ethernet controller: Intel Corporation Device 1559 (rev 04)
> 00: 86 80 59 15 00 00 10 00 04 00 00 02 00 00 00 00
> 10: 00 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00
> 20: 00 00 00 00 00 00 00 00 00 00 00 00 86 80 54 20
> 30: 00 00 00 00 c8 00 00 00 00 00 00 00 05 01 00 00
> 
> Inside  my  DomU I added code to print the PCI configuration registers
> and what I get after restarting the DomU is:
> 
> (d18) 14:57:04.042 src/e1000e.c@00150: 00: 86 80 59 15 00 00 10 00 04 00 00 
> 02 00 00 00 00
> (d18) 14:57:04.042 src/e1000e.c@00150: 10: 00 00 d0 f7 00 c0 d3 f7 81 f0 00 
> 00 00 00 00 00
> (d18) 14:57:04.042 src/e1000e.c@00150: 20: 00 00 00 00 00 00 00 00 00 00 00 
> 00 86 80 54 20
> (d18) 14:57:04.043 src/e1000e.c@00150: 30: 00 00 00 00 c8 00 00 00 00 00 00 
> 00 14 01 00 00
> (d18) 14:57:04.043 src/e1000e.c@00324: Enable PCI Memory Access
> (d18) 14:57:05.043 src/e1000e.c@00150: 00: 86 80 59 15 03 00 10 00 04 00 00 
> 02 00 00 00 00
> (d18) 14:57:05.044 src/e1000e.c@00150: 10: 00 00 d0 f7 00 c0 d3 f7 81 f0 00 
> 00 00 00 00 00
> (d18) 14:57:05.044 src/e1000e.c@00150: 20: 00 00 00 00 00 00 00 00 00 00 00 
> 00 86 80 54 20
> (d18) 14:57:05.045 src/e1000e.c@00150: 30: 00 00 00 00 c8 00 00 00 00 00 00 
> 00 14 01 00 00
> 
> As  you can see the pci configuration read from the pci-back driver by
> my DomU is different to the data in the Dom0 pci configuration!
> 
> Just  before  leaving my DomU I disable the pci memory access and this
> is what I see
> 
> (d18) 15:01:02.051 src/e1000e.c@00150: 00: 86 80 59 15 03 00 10 00 04 00 00 
> 02 00 00 00 00
> (d18) 15:01:02.051 src/e1000e.c@00150: 10: 00 00 d0 f7 00 c0 d3 f7 81 f0 00 
> 00 00 00 00 00
> (d18) 15:01:02.051 src/e1000e.c@00150: 20: 00 00 00 00 00 00 00 00 00 00 00 
> 00 86 80 54 20
> (d18) 15:01:02.052 src/e1000e.c@00150: 30: 00 00 00 00 c8 00 00 00 00 00 00 
> 00 14 01 00 00
> (d18) 15:01:02.052 src/e1000e.c@00541: Disable PCI Memory Access
> (d18) 15:01:02.052 src/e1000e.c@00150: 00: 86 80 59 15 00 00 10 00 04 00 00 
> 02 00 00 00 00
> (d18) 15:01:02.052 src/e1000e.c@00150: 10: 00 00 d0 f7 00 c0 d3 f7 81 f0 00 
> 00 00 00 00 00
> (d18) 15:01:02.052 src/e1000e.c@00150: 20: 00 00 00 00 00 00 00 00 00 00 00 
> 00 86 80 54 20
> (d18) 15:01:02.053 src/e1000e.c@00150: 30: 00 00 00 00 c8 00 00 00 00 00 00 
> 00 14 01 00 00
> 
> As  you  can  see the data is consistent with just writing 0000 to the
> pci control register.
> 
> This is the output from the debug version of the xen-pciback module.
> 
> [ 5429.351231] pciback 0000:00:19.0: enabling device (0000 -> 0003)
> [ 5429.351367] xen: registering gsi 20 triggering 0 polarity 1
> [ 5429.351373] Already setup the GSI :20
> [ 5429.351387] pciback 0000:00:19.0: xen-pciback[0000:00:19.0]: #20 on  
> disable-> enable
> [ 5429.351436] pciback 0000:00:19.0: xen-pciback[0000:00:19.0]: #20 on  
> enabled
> [ 5434.360078] pciback 0000:00:19.0: xen-pciback[0000:00:19.0]: #20 off  
> enable-> disable
> [ 5434.360116] pciback 0000:00:19.0: xen-pciback[0000:00:19.0]: #0 off  
> disabled
> [ 5434.361491] xen-pciback pci-20-0: fe state changed 5
> [ 5434.362473] xen-pciback pci-20-0: fe state changed 6
> [ 5434.363540] xen-pciback pci-20-0: fe state changed 0
> [ 5434.363544] xen-pciback pci-20-0: frontend is gone! unregister device
> [ 5434.467359] pciback 0000:00:19.0: resetting virtual configuration space
> [ 5434.467376] pciback 0000:00:19.0: free-ing dynamically allocated virtual 
> configuration space fields
> 
> Does this make any sense to you?

There was a bug in Xen pcibackend that I thought I upstreamed which could
be releated. It was not restoring the right registers to the PCI-device.

They are attached.

> 
> -- 
> Best regards,
>  Simon                            mailto:furryfuttock@xxxxxxxxx
>

Attachment: 0001-xen-pciback-Don-t-deadlock-when-unbinding.patch
Description: Text document

Attachment: 0006-xen-pciback-Restore-configuration-space-when-detachi.patch
Description: Text document

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel