Re: PCI pass-through problem for SN570 NVME SSD

["G.R." <firemeteor@xxxxxxxxxxxxxxxxxxxxx>]

On Mon, Jul 4, 2022 at 11:15 PM G.R. <firemeteor@xxxxxxxxxxxxxxxxxxxxx> wrote:
>
> On Mon, Jul 4, 2022 at 10:51 PM G.R. <firemeteor@xxxxxxxxxxxxxxxxxxxxx> wrote:
> >
> > On Mon, Jul 4, 2022 at 9:09 PM Roger Pau Monné <roger.pau@xxxxxxxxxx> wrote:
> > > Can you paste the lspci -vvv output for any other device you are also
> > > passing through to this guest?
> > >
>
> As reminded by this request, I tried to assign this nvme device to
> another FreeBSD12 domU.
Just to clarify, this time this NVME SSD is the only device I passed to this VM.

> This time it does not fail at the VM setup stage, but the device is
> still not usable at the domU.
> The nvmecontrol command is not able to talk to the device at all:
> nvme0: IDENTIFY (06) sqid:0 cid:0 nsid:0 cdw10:00000001 cdw11:00000000
> nvme0: ABORTED - BY REQUEST (00/07) sqid:0 cid:0 cdw0:0
> nvme0: IDENTIFY (06) sqid:0 cid:0 nsid:0 cdw10:00000001 cdw11:00000000
> nvme0: ABORTED - BY REQUEST (00/07) sqid:0 cid:0 cdw0:0
>
> The QEMU log says the following:
> 00:05.0] Write-back to unknown field 0x09 (partially) inhibited (0x00)
> [00:05.0] If the device doesn't work, try enabling permissive mode
> [00:05.0] (unsafe) and if it helps report the problem to xen-devel
> [00:05.0] msi_msix_setup: Error: Mapping of MSI-X (err: 61, vec: 0x30, entry 
> 0)

I retried with the following:
pci=['05:00.0,permissive=1,msitranslate=1']
Those extra options suppressed some error logging, but still didn't
make the device usable to the domU.
The nvmecontrol command still get ABORTED result from the kernel...

The only thing remained in the QEMU file is this one:
[00:05.0] msi_msix_setup: Error: Mapping of MSI-X (err: 61, vec: 0x30, entry 0)

The xl dmesg appears to be identical except this line is gone:
(XEN) d[IO]: assign (0000:05:00.0) failed (-16)
And in both case I see the following, which suggests that the MSI-X
failure is worked-around.
(XEN) d5: bind: m_gsi=16 g_gsi=36 dev=00.00.5 intx=0

What's the current situation now?