Re: [Xen-users] VGA Passthrough attempted with an Nvidia GTX970

[Peter Kay <syllopsium@xxxxxxxxxxxxxxxx>]

On 12 February 2015 17:19:05 GMT+00:00, Gordan Bobic <gordan@xxxxxxxxxx> wrote:
>On 2015-02-12 12:30, Peter Kay wrote:
>> You don't want to know about the BIOS bugs in my (obscure) hardware,
>> or the fact the embedded graphics card shares a PCI-e IOMMU group
>with
>> the southbridge PCI-e slot, the only one useful in this case. Of
>> course Xen doesn't tell you that, KVM has to be used.
>
>What is an IOMMU "group" and why does it matter?
See http://vfio.blogspot.co.uk/2014/08/iommu-groups-inside-and-out.html

An IOMMU group is a set of devices attached to the same root
port/upstream without being protected from transfers between each
other by the IOMMU/supporting chipset. Often all the 32 bit PCI slots
are like this. If devices in the same group are passed to different
VMs there is the possibility of one VM affecting the other.

There is (I think) now a standard to query this functionality but most
is by a set of quirks (Intel/AMD say which chipsets are safe based on
their IDs), then incorporated into VM systems like Xen. My system is
very old, so nothing with the same root port is protected from each
other.

The upshot is that without this you will e.g. get static on the
soundcard that is using the same PCI bus as a NIC. When I successfully
passed through the Quadro(ish) 6000 on my system, the console driven
by an embedded Matrox G200e went black, because it shared the same
IOMMU group.

When KVM is installed, there's a load of iommu_group entries under
/sys. One for the actual list of groups (iommu_groups directory) and
another under each device (iommu_group directory), showing which
devices it shares a group with i.e.

0000:00:06.0
0000:02:00.0

The full list on my completely populated system is (this is taken from
a combination of ls /sys/kernel/iommu_groups and subdirectories. plus
cut and paste from lspci)

1
0000:00:01.0@ 00:01.0 PCI bridge: Intel Corporation 3200/3210 Chipset
Host-Primary PCI Express Bridge
0000:01:00.0@ 01:00.0 VGA compatible controller: AMD/ATI [Advanced
Micro Devices, Inc.] Caicos [Radeon HD 6450]
0000:01:00.1@ 01:00.1 Audio device: AMD/ATI [Advanced Micro Devices,
Inc.] Caicos HDMI Audio [Radeon HD 6400 Series]
2
0000:00:06.0@ 00:06.0 PCI bridge: Intel Corporation 3210 Chipset
Host-Secondary PCI Express Bridge
0000:02:00.0@ 02:00.0 USB controller: Etron Technology, Inc.
EJ188/EJ198 USB 3.0 Host Controller
3
0000:00:19.0@ 00:19.0 Ethernet controller: Intel Corporation 82566DM-2
Gigabit Network Connection (rev 02)
4
0000:00:1a.0@ 00:1a.0 USB controller: Intel Corporation 82801I (ICH9
Family) USB UHCI Controller #4 (rev 02)
0000:00:1a.1@ 00:1a.1 USB controller: Intel Corporation 82801I (ICH9
Family) USB UHCI Controller #5 (rev 02)
0000:00:1a.7@ 00:1a.7 USB controller: Intel Corporation 82801I (ICH9
Family) USB2 EHCI Controller #2 (rev 02)
5
0000:00:1c.0@ 00:1c.0 PCI bridge: Intel Corporation 82801I (ICH9
Family) PCI Express Port 1 (rev 02)
0000:00:1c.4@ 00:1c.4 PCI bridge: Intel Corporation 82801I (ICH9
Family) PCI Express Port 5 (rev 02)
0000:03:00.0@ 03:00.0 VGA compatible controller: NVIDIA Corporation
GF100GL [Quadro 6000] (rev a3)
0000:03:00.1@ 03:00.1 Audio device: NVIDIA Corporation GF100 High
Definition Audio Controller (rev a1)
0000:04:00.0@ 04:00.0 VGA compatible controller: Matrox Electronics
Systems Ltd. MGA G200e [Pilot] ServerEngines (SEP1) (rev 02)
6
0000:00:1d.0@ 00:1d.0 USB controller: Intel Corporation 82801I (ICH9
Family) USB UHCI Controller #1 (rev 02)
0000:00:1d.1@ 00:1d.1 USB controller: Intel Corporation 82801I (ICH9
Family) USB UHCI Controller #2 (rev 02)
0000:00:1d.2@ 00:1d.2 USB controller: Intel Corporation 82801I (ICH9
Family) USB UHCI Controller #3 (rev 02)
0000:00:1d.7@ 00:1d.7 USB controller: Intel Corporation 82801I (ICH9
Family) USB2 EHCI Controller #1 (rev 02)
7
0000:00:1e.0@ 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 92)
0000:05:00.0@ 05:00.0 Multimedia audio controller: Creative Labs
SB0400 Audigy2 Value
0000:05:01.0@ 05:01.0 FireWire (IEEE 1394): Texas Instruments TSB82AA2
IEEE-1394b Link Layer Controller (rev 01)
0000:05:02.0@ 05:02.0 Ethernet controller: Intel Corporation 82541GI
Gigabit Ethernet Controller (rev 05)
8
0000:00:1f.0@ 00:1f.0 ISA bridge: Intel Corporation 82801IR (ICH9R)
LPC Interface Controller (rev 02)
0000:00:1f.2@ 00:1f.2 SATA controller: Intel Corporation 82801IR/IO/IH
(ICH9R/DO/DH) 6 port SATA Controller [AHCI mode] (rev 02)
0000:00:1f.3@ 00:1f.3 SMBus: Intel Corporation 82801I (ICH9 Family)
SMBus Controller (rev 02)

From the above you can see why I had the issues with static and a
blanking console screen
(and no, the Radeon 6450 does not work, hence the other Quadro on order)

>Q2000 has half the TDP and is considerably shorter. As for thickness,
>they are both single slot cards, so I doubt there is a worthwhile
>difference.

It is, but some models of the Q2000 have extra heat sinks on the back
of the card, and it's that tight (close to a CPU cooler) I can't risk
it. I've also bought some PCIe slot ribbon cable extenders as a backup
if it won't squeeze in. I have plenty of cooling and a beefy PSU.

PK

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users