Re: [Xen-users] PCI passthrough of HP P420i RAID controller

[jacek burghardt <jaceksburghardt@xxxxxxxxx>]

I Played with P800 and it was locking up with linux also. I had used couple other brands of raid controllers and I was able to assign them to linux and they worked without an issue. I haveÂPromise Technology, Inc. 81384 Âthat i was able to assign to linux using pv.

On Tue, Jul 8, 2014 at 8:13 AM, Simon Waterman <simon.waterman@xxxxxxxxxxx> wrote:

Hi,

On Mon, Jul 7, 2014 at 15:11, jacek burghardt <jaceksburghardt@xxxxxxxxx> wrote:
>> I had passed hp sas to windows 2012 R2 and it works fine. i had tried to assign
>> it to Linux but the drivers were setting the card wrng and causing lockup requiring
>> me to reboot server.

Its interesting to know that it worked fine with the Windows driver and that you got a
lockup too under Linux. ÂWas it the same P420i controller with SAS drives?

I'm not sure its the Linux hpsa driver causing the lockup though.

I spent some time trying to diagnose where in the driver the lockup was being caused
and I tried modifying the probe code in the driver domain to reject the device immediately
before doing anything with it. ÂWhen I tried to detach the device from the driver domain
and re-bind it to the hpsa driver in the Dom-0 it failed in the same way despite never
being touched by the hpsa driver in the driver domain.

Purely unbinding the device from the hpsa driver and binding it to xen-pciback doesn't
seem to cause the problem because if I rebind it to hpsa without doing the attach to
the driver domain the device appears to function normally. ÂIn other words, binding and
unbinding from xen-pciback seems fine - attaching the device to the driver domain
causes the problem even when the hpsa driver isn't in the picture on the driver domain.

Best wishes,

Simon

On Mon, Jul 7, 2014 at 7:34 AM, Simon Waterman Â<simon.waterman@xxxxxxxxxxx> wrote:
ÂHi,

We've been successfully using Xen with PCI pass-through of disk and
network cards for a couple of years now. ÂWe have this architecture working
successfully with a wide range of hardware from HP and Dell but recently
have been getting problems with the HP ProLiant DL380p Gen8 and its
P420i RAID controller.

When we bind the RAID controller to pciback and issue the appropriate
'xl pci-attach' command the controller appears on the driver domain but fails
during probing by the hpsa driver. ÂInterestingly, in about 1 in every 10
test runs the hpsa driver probes successfully, the controller appears in the
driver domain and the disks appear to function correctly.

In the failure cases the syslog output from the hpsa driver is as follows:

Jul Â7 13:30:01 zylevel0-HVM-domU kernel: [ Â 54.855597] bus: 'pci': driver_prob
e_device: matched device 0000:00:00.0 with driver hpsa
Jul Â7 13:30:01 zylevel0-HVM-domU kernel: [ Â 54.855599] bus: 'pci': really_prob
e: probing driver hpsa with device 0000:00:00.0
Jul Â7 13:30:01 zylevel0-HVM-domU kernel: [ Â 54.855605] HP HPSA Driver (v 3.4.4
-1)
Jul Â7 13:30:01 zylevel0-HVM-domU kernel: [ Â 54.855684] hpsa 0000:00:00.0: enab
ling device (0000 -> 0003)
Jul Â7 13:30:01 zylevel0-HVM-domU kernel: [ Â 54.855983] hpsa 0000:00:00.0: Xen
PCI mapped GSI34 to IRQ29
Jul Â7 13:30:01 zylevel0-HVM-domU kernel: [ Â 54.856089] hpsa 0000:00:00.0: enab
ling bus mastering
Jul Â7 13:30:01 zylevel0-HVM-domU kernel: [ Â 54.856659] hpsa 0000:00:00.0: MSIX
Jul Â7 13:30:01 zylevel0-HVM-domU kernel: [ Â 54.857384] hpsa 0000:00:00.0: Logi
cal aborts not supported
Jul Â7 13:40:00 zylevel0-HVM-domU kernel: [ Â654.318297] hpsa 0000:00:00.0: unable
to get board into simple mode
Jul Â7 13:40:00 zylevel0-HVM-domU kernel: [ Â654.318478] hpsa: probe of 0000:00:00.0
rejects match -19

The error seems to originate from the hpsa_enter_simple_mode function and looks
like the first point where an interaction with the board via IO mapped memory is
expected to result in a response from the board.

We are using Xen 4.4 and a newly built Linux 3.15.0 kernel with Ubuntu trusty. ÂVTd is
disabled in the BIOS of the Dom-0. ÂWe use the following Grub commands for the Dom-0 boot:

    multiboot    /boot/xen.gz placeholder dom0_mem=max:2048m iommu=0
loglvl=all guest_loglvl=all
    module Â/boot/vmlinuz-3.15.0 placeholder root=UUID=f56cf02b-d46f-4e2a-b667-242f97ec82f5
ro Âdebug
    module Â--nounzip  /boot/initrd.img-3.15.0

The driver domain is a PV guest running the same kernel version as the Dom-0.
The Xen configuration file contains the following kernel boot parameters:

extra="earlyprintk=xen debug max_loop=64 iommu=soft"

After we issue the pci-attach command the following additional lines appear in the
'xl dmesg' log:

(XEN) grant_table.c:289:d0 Increased maptrack size to 2 frames
(XEN) mm.c:809: d2: Forcing read-only access to MFN f7f02
(XEN) mm.c:809: d0: Forcing read-only access to MFN f7f02

We were wondering whether anyone in the community had got PCI pass-through of
the P420i working, or any pointers of where to look to further diagnose the problem.

Best wishes,

Simon

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users

 Â

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users