Re: [Xen-devel] AMD_IOV: IO_PAGE_FALT trying to pass through Mellanox ConnectX HCA (debian testing)

[Ward Vandewege <ward@xxxxxxx>]

On Fri, Jan 28, 2011 at 02:27:42PM -0500, Konrad Rzeszutek Wilk wrote:
> > The mlx4_core driver in the domU is not happy:
> > 
> > [    0.411867] mlx4_core: Mellanox ConnectX core driver v0.01 (May 1, 2007)
> > [    0.411879] mlx4_core: Initializing 0000:00:00.0
> > [    0.412027] mlx4_core 0000:00:00.0: enabling device (0000 -> 0002)
> > [    0.412027] mlx4_core 0000:00:00.0: Xen PCI enabling IRQ: 19
> > [    1.417477] mlx4_core 0000:00:00.0: Installed FW has unsupported command
> > interface revision 0.
> > [    1.417509] mlx4_core 0000:00:00.0: (Installed FW version is 0.0.000)
> > [    1.417527] mlx4_core 0000:00:00.0: This driver version supports only
> > revisions 2 to 3.
> > [    1.417549] mlx4_core 0000:00:00.0: QUERY_FW command failed, aborting.
> > 
> > When trying to boot a PV domU with kernel options iommu=soft and
> > swiotlb=force, the output is slightly different. The full bootlog is 
> > attached
> 
> Don't use swiotlb=force unless neccessary.

OK; I just tried it without (for PV), same result:

[    0.420448] mlx4_core: Mellanox ConnectX core driver v1.0-ofed1.5.2
(August 4, 2010)
[    0.420462] mlx4_core: Initializing 0000:00:00.0
[    0.420804] mlx4_core 0000:00:00.0: enabling device (0000 -> 0002)
[    0.421477] mlx4_core 0000:00:00.0: Xen PCI enabling IRQ: 19
[    1.429824] mlx4_core 0000:00:00.0: Installed FW has unsupported command
interface revision 0.
[    1.429858] mlx4_core 0000:00:00.0: (Installed FW version is 0.0.000)
[    1.429876] mlx4_core 0000:00:00.0: This driver version supports only
revisions 2 to 3.
[    1.429895] mlx4_core 0000:00:00.0: QUERY_FW command failed, aborting.

and

(XEN) AMD_IOV: IO_PAGE_FALT: domain:2, device id:0x200, fault
address:0x2c03000
(XEN) AMD_IOV: IO_PAGE_FALT: domain:2, device id:0x200, fault
address:0x2c03040
(XEN) AMD_IOV: IO_PAGE_FALT: domain:2, device id:0x200, fault
address:0x2c03080
(XEN) AMD_IOV: IO_PAGE_FALT: domain:2, device id:0x200, fault
address:0x2c030c0

> That is b/c you are missing iommu=pv on Xen hypervisor line, and

Hmm, I do have it:

  # xm dmesg |grep iommu
  (XEN) Command line: placeholder iommu=pv,verbose,amd_iommu_debug

But maybe it's not being picked up?

> you might need to make sure your driver is using the VM_IO flag.
> 
> There was some discussion on LKML about this and they proposed
> a patch that wasn't neccessary. Don't remember the details but I can
> look that up next week.

Do you mean this thread?

  
http://xen.1045712.n5.nabble.com/Infiniband-from-userland-in-dom0-process-killed-bad-pagetable-td3259124.html

Thanks,
Ward.


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel