Re: PCI pass-through problem for SN570 NVME SSD

[Jan Beulich <jbeulich@xxxxxxxx>]

On 05.07.2022 13:31, G.R. wrote:
> On Tue, Jul 5, 2022 at 5:04 PM Jan Beulich <jbeulich@xxxxxxxx> wrote:
>>
>> On 04.07.2022 18:31, G.R. wrote:
>>> On Tue, Jul 5, 2022 at 12:21 AM Roger Pau Monné <roger.pau@xxxxxxxxxx> 
>>> wrote:
>>>>> I retried with the following:
>>>>> pci=['05:00.0,permissive=1,msitranslate=1']
>>>>> Those extra options suppressed some error logging, but still didn't
>>>>> make the device usable to the domU.
>>>>> The nvmecontrol command still get ABORTED result from the kernel...
>>>>>
>>>>> The only thing remained in the QEMU file is this one:
>>>>> [00:05.0] msi_msix_setup: Error: Mapping of MSI-X (err: 61, vec: 0x30, 
>>>>> entry 0)
>>>>
>>>> Hm it seems like Xen doesn't find the position of the MSI-X table
>>>> correctly, given there's only one error path from msi.c returning
>>>> -ENODATA (61).
>>>>
>>>> Are there errors from pciback when this happens?  I would expect the
>>>> call to pci_prepare_msix() from pciback to fail and thus also report
>>>> some error?
>>>>
>>>> I think it's likely I will have to provide an additional debug patch
>>>> to Xen, maybe Jan has an idea of what could be going on.
>>>>
>>> pciback reports the same MSI-x related error.
>>> But even with DEBUG enabled, I didn't see more context reported.
>>> Please find details from the attachment.
>>
>> And nothing pertinent in "xl dmesg"? Looking back through the thread I
>> couldn't spot a complete hypervisor log (i.e. from boot to assignment
>> attempt). An issue with MSI-X table determination, as Roger suspects,
>> would typically be associated with a prominent warning emitted to the
>> log. But there are also further possible sources of -ENXIO, which
>> would go silently.
> Please find the xl dmesg in the attachment.
> It's with the two FreeBSD domU attempts so it should have captured
> some culprits if there is any...

Nothing useful in there. Yet independent of that I guess we need to
separate the issues you're seeing. Otherwise it'll be impossible to
know what piece of data belongs where.

Jan