[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: PCI pass-through problem for SN570 NVME SSD
- To: "G.R." <firemeteor@xxxxxxxxxxxxxxxxxxxxx>
- From: Roger Pau Monné <roger.pau@xxxxxxxxxx>
- Date: Mon, 4 Jul 2022 18:05:43 +0200
- Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com; dkim=pass header.d=citrix.com; arc=none
- Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=5c3a6rVImi5dqRSIfLU7WycXTqelQQ5pk1VTTDgZsQk=; b=YJovUrifQzkS/vXWhOEPLe/jYKldDb+xEiQbjI4IRoz0wFQzjxn9Zrq7q68FpqztFqXv8zyhud4dTMkVxjftnNP1SfCA1ETUcafTt0X12Ees9WK5G8e1EvSgAokbxVPvM9OEDLpPoNHKUunF98yb+9mI/qw5kGnqfTNkGp14baun/vpCTXDa2c5uuNj7mXG4HLSNMAM4xEYAqyvstmWjt9M3VK6NvwPJzBk46ufOQwU8602GGr9q1IhVOZ6Jo/U6s7V1j3rt4GK4GjjQJ4V/onDe0f0AAkxSoi++m7+VLx8/Q+F88UvKHo3F8zvJ2glVHQO/isvYpymFljDcQSuvYQ==
- Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WFUjRUF7NGzRiCg/hS5KW+ihiEEOjiOHWT9E3GnpFTXmE4WiKbXt0vbL5A3+Uq8MKw5cgx9lE7zFKQuR4AuLgFE74VEP0glKui0zIyd8ypYrywLmeDu8+GdcD6dtYU/3A1wTum/ozpJoe2kPke7Zce7yvXIb9qjrtL6ESMnNL08PhCky3GCMCP2ZXvxbYhGzOjmD3oe8wIbJTQ/1r5/gshkZEOQijBouSCxAUaFCWs5y6UQN+BsYo6oViQYuSMtssjYGBAA/VK8fhwQHQa45TLYrMjUKUYAnO3VMqR4E925jZpo5yyq6HBUL0qqa/540k2PG9H698+69C+rojwZ+og==
- Authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=citrix.com;
- Cc: xen-devel <xen-devel@xxxxxxxxxxxxx>, Jan Beulich <jbeulich@xxxxxxxx>
- Delivery-date: Mon, 04 Jul 2022 16:06:03 +0000
- Ironport-data: A9a23:M3dGOK1nxaSdZD+akPbD5alwkn2cJEfYwER7XKvMYLTBsI5bpz1Rn DQdXDiHbv6LMTHwKYwlaISz9BhSvZKAzoAySlRlpC1hF35El5HIVI+TRqvS04J+DSFhoGZPt Zh2hgzodZhsJpPkjk7xdOCn9xGQ7InQLlbGILes1htZGEk1Ek/NtTo5w7Rj2tEw2oDja++wk YiaT/P3aQfNNwFcagr424rbwP+4lK2v0N+wlgVWicFj5DcypVFMZH4sDfjZw0/DaptVBoaHq 9Prl9lVyI97EyAFUbtJmp6jGqEDryW70QKm0hK6UID66vROS7BbPg/W+5PwZG8O4whlkeydx /1VqbKragF3DJeTp8dNTABYFj91G7JZreqvzXiX6aR/zmXgWl60mbBEKhhzOocVvOFqHWtJ6 PoUbigXaQyOjP63x7T9TfRwgsMkL4/gO4Z3VnNIlGmFS6p5B82aBfmVjTNb9G5YasRmB/HRa tBfcTNyRB/BfwdOKhEcD5dWcOKA2SWkLWAG8A/9Sawf42b8yAp/7bLWH/X8X/urVMtLlHnGj zeTl4j+KlRAXDCF8hKL82ihg+LTkCThcJ8JGaejsOVtnUeYy2IUEhIbE122vZGRmkO4Ht5SN UEQ0i4vtrQpslymSJ/6RRLQiHyOswMYWtFQO/Yn8wzLwa3Rizt1HUABRz9FLdYg7ck/QGR10 kfTx4+2QztyrLeSVHSRsK+Oqi+/MjQUKmlEYjIYSQwC4J/op4RbYg/zc+uP2ZWd1rXdcQwcC RjTxMTir93/VfI26pg=
- Ironport-hdrordr: A9a23:CqVvrquAxM6lRvc0p3KwBLr17skC7YMji2hC6mlwRA09TyXGra 2TdaUgvyMc1gx7ZJhBo7+90We7MBbhHLpOkPEs1NCZLXLbUQqTXfhfBO7ZrwEIdBefygcw79 YCT0E6MqyLMbEYt7eE3ODbKadG/DDvysnB64bjJjVWPGdXgslbnntE422gYylLrWd9dPgE/M 323Ls7m9PsQwVgUu2LQl0+G8TTrdzCk5zrJTYAGh4c8QGLyRel8qTzHRS01goXF2on+8ZrzU H11yjCoomzufCyzRHRk0fV8pRtgdPkjv9OHtaFhMQ5IijlziyoeINicbufuy1dmpDn1H8a1P 335zswNcV67H3cOkmzvBvWwgHllA0j7nfzoGXo9EfLkIjcfnYXGsBBjYVWfl/y8Ew7puxx16 pNwiawq4dXJQmoplWw2/H4EzVR0makq3srluAey1ZFV5EFVbNXpYsDuGtIDZY7Gj7g4oxPKp giMCjl3ocZTbqmVQGZgoE2q+bcHkjbXy32CHTqg/blnAS/xxtCvgglLM92pAZzyHtycegH2w 3+CNUYqFh/dL5pUUtDPpZwfSKWMB24ffueChPkHX3XUIc6Blnql7nbpJ0I2cDCQu168HJ1ou WLbG9l
- List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
On Mon, Jul 04, 2022 at 11:37:13PM +0800, G.R. wrote:
> On Mon, Jul 4, 2022 at 11:15 PM G.R. <firemeteor@xxxxxxxxxxxxxxxxxxxxx> wrote:
> >
> > On Mon, Jul 4, 2022 at 10:51 PM G.R. <firemeteor@xxxxxxxxxxxxxxxxxxxxx>
> > wrote:
> > >
> > > On Mon, Jul 4, 2022 at 9:09 PM Roger Pau Monné <roger.pau@xxxxxxxxxx>
> > > wrote:
> > > > Can you paste the lspci -vvv output for any other device you are also
> > > > passing through to this guest?
> > > >
> >
> > As reminded by this request, I tried to assign this nvme device to
> > another FreeBSD12 domU.
> Just to clarify, this time this NVME SSD is the only device I passed to this
> VM.
>
> > This time it does not fail at the VM setup stage, but the device is
> > still not usable at the domU.
> > The nvmecontrol command is not able to talk to the device at all:
> > nvme0: IDENTIFY (06) sqid:0 cid:0 nsid:0 cdw10:00000001 cdw11:00000000
> > nvme0: ABORTED - BY REQUEST (00/07) sqid:0 cid:0 cdw0:0
> > nvme0: IDENTIFY (06) sqid:0 cid:0 nsid:0 cdw10:00000001 cdw11:00000000
> > nvme0: ABORTED - BY REQUEST (00/07) sqid:0 cid:0 cdw0:0
> >
> > The QEMU log says the following:
> > 00:05.0] Write-back to unknown field 0x09 (partially) inhibited (0x00)
> > [00:05.0] If the device doesn't work, try enabling permissive mode
> > [00:05.0] (unsafe) and if it helps report the problem to xen-devel
> > [00:05.0] msi_msix_setup: Error: Mapping of MSI-X (err: 61, vec: 0x30,
> > entry 0)
>
> I retried with the following:
> pci=['05:00.0,permissive=1,msitranslate=1']
> Those extra options suppressed some error logging, but still didn't
> make the device usable to the domU.
> The nvmecontrol command still get ABORTED result from the kernel...
>
> The only thing remained in the QEMU file is this one:
> [00:05.0] msi_msix_setup: Error: Mapping of MSI-X (err: 61, vec: 0x30, entry
> 0)
Hm it seems like Xen doesn't find the position of the MSI-X table
correctly, given there's only one error path from msi.c returning
-ENODATA (61).
Are there errors from pciback when this happens? I would expect the
call to pci_prepare_msix() from pciback to fail and thus also report
some error?
I think it's likely I will have to provide an additional debug patch
to Xen, maybe Jan has an idea of what could be going on.
Roger.
|
