Xen 4.14, mpt3sas + HBA 9400-16i

[Holger Schramm <lists@xxxxxxxxxx>]

Hi folks,

I am struggling with Xen 4.14 on Debian Bullseye to get an Broadcom HBA 
9400-16i Controller working. Used Kernel: Debian Standard 5.10.0-13 and 
a self baked one as of 5.15.35 with derived default config from debian.
If I boot the system in normal mode, without Xen everything works fine.
As soon as I am booting into Xen, the Problems are starting.

The boot process floods the dmesg with

[  940.815488] mpt3sas_cm0: sending diag reset !!
[  941.548926] mpt3sas_cm0: diag reset: SUCCESS
[  941.799450] mpt3sas_cm0: CurrentHostPageSize is 0: Setting default 
host page size to 4k
[  941.931194] mpt3sas_cm0: _base_display_fwpkg_version: complete
[  941.931756] mpt3sas_cm0: FW Package Ver(19.00.00.00)
[  956.936874] mpt3sas_cm0: config_request: manufacturing(0), action(0), 
form(0x00000000), smid(6140)
[  956.937394] mpt3sas_cm0: _config_request: command timeout
[  956.937819] mpt3sas_cm0: Command Timeout

[ 1017.358576] mpt3sas_cm0: _config_request: attempting retry (2)
[ 1032.455458] mpt3sas_cm0: config_request: manufacturing(11), 
action(0), form(0x00000000), smid(6140)
[ 1032.455959] mpt3sas_cm0: _config_request: command timeout
[ 1032.456375] mpt3sas_cm0: Command Timeout
[ 1032.456788] mf:

[ 1032.456790] 04000000
[ 1032.457581] 00000000
[ 1032.457977] 00000000
[ 1032.458372] 00000000
[ 1032.458737] 00000000
[ 1032.459084] 090b0000
[ 1032.459487] 00000000
[ 1032.459708] d3000000
[ 1032.459857]

[ 1032.460005] ffffffff
[ 1032.460299] ffffffff
[ 1032.460450] 00000000


I tried to set the msix to disabled via modprobe.d


/etc/modprobe.d/mpt3sas.conf:
----------- >8 ---------------

options mpt3sas msix_disable=1

----------- 8< ---------------

but this doesn't either work.

Then i tried to install the driver directly from broadcom but this 
doesn't work too. I got another error:
[   33.856072] mpt3sas_cm0: failure at 
/var/lib/dkms/mpt3sas/41.00.00.00/build/mpt3sas_scsih.c:16229/_scsih_probe()!
In all states of "error" I am not able to connect to the HBA via 
storcli, except in normal mode. If I try it wile running xen:

CLI Version = 007.1613.0000.0000 Oct 29, 2020
Operating system = Linux 5.10.0-13-amd64
Status Code = 0
Status = Success
Description = None

Number of Controllers = 0
Host Name = xm1010-neu
Operating System  = Linux 5.10.0-13-amd64
StoreLib IT Version = 07.1700.0200.0000
StoreLib IR3 Version = 16.14-0

storcli.log

Mon Apr 25 23:43:17.159 2022 : CtMain::main processing for command=show
Mon Apr 25 23:43:17.162 2022 : Entering MWCliRootCp::process 
ver=007.2103.0000.0000 Dec 08, 2021 copyright=2021
Mon Apr 25 23:43:17.162 2022 : Entering ShowSystemCp::process
Mon Apr 25 23:43:17.162 2022 : Entering LinuxControllerExplorer::explore
Mon Apr 25 23:43:17.162 2022 : Entering 
LinuxControllerExplorer::getCtrlCount
Mon Apr 25 23:43:17.163 2022 : LinuxControllerExplorer::getCtrlCount 
_hDvr=-1 _hDvrSwr =-1
Mon Apr 25 23:43:17.163 2022 : LinuxControllerExplorer::explore ctrlCount=1
Mon Apr 25 23:43:17.163 2022 : returning 
LinuxControllerExplorer::explore ctrlIdListPtr
Mon Apr 25 23:43:17.163 2022 : SlItControllerExplorer::explore - Entering
Mon Apr 25 23:43:17.163 2022 : StorelibItDp::processLibCommand SL 
request sent
Mon Apr 25 23:43:17.165 2022 : StorelibItDp::processLibCommand SL 
responded with retcode=0
Mon Apr 25 23:43:17.165 2022 : StorelibItDp::processLibCommand SL 
request sent
Mon Apr 25 23:43:17.165 2022 : StorelibItDp::processLibCommand SL 
responded with retcode=0
Mon Apr 25 23:43:17.165 2022 : SlItControllerExplorer::explore - Getting 
the StoreLIB instance for SAS3.5 IT controller
Mon Apr 25 23:43:17.165 2022 : StorelibItDp::processLibCommand SL 
request sent
Mon Apr 25 23:43:17.206 2022 : StorelibItDp::processLibCommand SL 
responded with retcode=0
Mon Apr 25 23:43:17.206 2022 : SlItControllerExplorer::explore - Issuing 
SL_GET_CTRL_LIST to SLIT
Mon Apr 25 23:43:17.206 2022 : StorelibItDp::processLibCommand SL 
request sent
Mon Apr 25 23:43:17.206 2022 : StorelibItDp::processLibCommand SL 
responded with retcode=0
Mon Apr 25 23:43:17.206 2022 : SlItControllerExplorer::explore - Issuing 
SL_GET_CTRL_LIST to SLIT - retCode:0x0
Mon Apr 25 23:43:17.206 2022 : SlItControllerExplorer::explore - 
Controller count:0
Mon Apr 25 23:43:17.206 2022 : LinuxMfiDataProvider::fireIoctl 
DCMD=1010000 mbox.w[0]=1 mbox.w[1]=0 mbox.w[2]=0
Mon Apr 25 23:43:17.206 2022 : LinuxMfiDataProvider::fireIoctl 
send_ioctl retcode=-1 cmdStatus=255
Mon Apr 25 23:43:17.206 2022 : LinuxMfiDataProvider::fireIoctl returning 
with =-1 errno=9
Mon Apr 25 23:43:17.206 2022 : the property ctrlBusy is set to 0
Mon Apr 25 23:43:17.206 2022 : Entering ShowSystemCp::ShowSystemSummary
Mon Apr 25 23:43:17.206 2022 : Entering ShowSystemCp::getHeadline
Mon Apr 25 23:43:17.206 2022 : StorelibItDp::processLibCommand SL 
request sent
Mon Apr 25 23:43:17.206 2022 : StorelibItDp::processLibCommand SL 
responded with retcode=0
Mon Apr 25 23:43:17.206 2022 : Entering ShowSystemCp::getSLITVersion = 
07.2103.0200.0000
Mon Apr 25 23:43:17.206 2022 : StorelibItDp::processLibCommand SL 
request sent
Mon Apr 25 23:43:17.206 2022 : StorelibItDp::processLibCommand SL 
responded with retcode=0
Mon Apr 25 23:43:17.207 2022 : Entering ShowSystemCp::getSLIR3Version = 
16.14-0
Mon Apr 25 23:43:17.207 2022 : MWCliRootCp::process processed for 
subtokenID=14 errorCode=0
Mon Apr 25 23:43:17.207 2022 : Entering MWCliRootCp::handleOuput 
retCode=0, DriveFull=0
Mon Apr 25 23:43:17.207 2022 : StorelibItDp::processLibCommand SL 
request sent
Mon Apr 25 23:43:17.207 2022 : StorelibItDp::processLibCommand SL 
responded with retcode=0
Mon Apr 25 23:43:17.207 2022 : StorelibItDp::processLibCommand SL 
request sent
Mon Apr 25 23:43:17.207 2022 : StorelibItDp::processLibCommand SL 
responded with retcode=0
Mon Apr 25 23:43:17.207 2022 : CtMain::main command completed with retcode=0

How can I get this controller working with xen?

Kind regards,

~Holger