[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-users] [Xen VGA Passthrough] AMD R9 290X GPU???

On 2014-09-12 15:24, Peter Kay wrote:
On 12 September 2014 14:46, Gordan Bobic <gordan@xxxxxxxxxx> wrote:

On 2014-09-12 13:14, Peter Kay wrote:

It's possible that the reason mixing multiple cards doesn't work
because ATI drivers allegedly try and initialise cards, even if
they're claimed elsewhere.


Most of my attempts so far have been with KVM. With that a HD6950
passes through just fine once a NoSnoop patch is applied, but
having a
low end Nvidia card in the host Linux breaks things badly.

Are you saying that passthrough of an ATI card causes the host
to crash even though the host is running an Nvidia card?

No. The lower end NVidia cards (GT210) have a (dom0/KVM host) Linux
driver that causes instability if passthrough is used, due to VGA
arbitration. Using the official NVidia driver is a bad idea. Nouveau
is slightly better iirc, but has other issues. Not sure if higher end
Nvidia cards fix this.

That's weird - until a couple of weeks ago I've been using an 8800GT
for my dom0 GPU (780Tis for domUs) and had no problems. Just made
sure that the xen-pciback was loaded and bound to the 780Tis before
the nvidia driver for the 8800GT card was loaded. Never had a
problem. Quadro 2000 also works, as did, IIRC, GTX470 and GTX480.

GT630 causes the system to lock-up about 80% of the time as soon
as Xorg starts. Same driver, same configuration. I am at a loss
to explain it.

I've never even tried using VGA passthrough, only secondary
PCI passthrough - I can live with the video output being confined
to the emulated GPU with VNC output and only getting output from
the GPU once the driver loads.

I haven't managed to get a 6950 working full stop, either primary or
secondary in Xen. It's fine in KVM.

That's odd. I had a HD6450 and HD7970 working in domU as well
as can be expected (works fine on the first domU boot, with
artifacting, slowdown, or just plain system-wide lock-up after
domU is rebooted or stopped and started again).

I wanted a passively cooled dom0 GPU and wanted to use the GT630,
but since that just wouldn't work, I opted for the HD6450. I don't
need that much GPU power for dom0, it's just that GT630 is faster
and has lower TDP than the HD6450 (and is 2-3 generations newer).

1) If your criteria is passthrough of any type, KVM is a better
than Xen. It works and it's also easily possible to identify
isolation groups, aiding stability.

I'm going to guess this requires IOMMU ACL to work (the correct
name/acronym for it escapes me at the moment) - which it doesn't
on some PCIe bridges (NF200), and is broken on many BIOS-es even
when the hardware itself isn't impossibly broken.

To be fair, you'll have just as many issues with KVM with a NF200
motherboard.. I think there are workarounds there, but I haven't been
keeping up.

The main problem is that I need to prevent the guest domains from
using the memory addresses within the guest memory map that are used
for PCI IOMEM regions. Ideally this should be done by presending the
e820 map that marks those regions as reserved. As a plan B, however,
marking the regions as bad/broken memory in the guest should work,
even if it does mean some of the RAM becomes unusable/wasted.

2) If you want AMD GPU passthrough, use KVM, it's solid and you'll
save yourself a huge amount of pain.

Guest reboots work without side effects?

Yes. It's solid.

Are you using a kernel that implements the bus reset to reset
the card? Or is KVM capable of side-loading the GPU BIOS and
re-POST-ing that for recent ATI cards (with VBIOS > 64KB)?

Last time I tested Xen was somewhat faster than KVM. I would
have preferred to use KVM because unlike the Xen dom0, with
KVM the host domain isn't running as a virtual domain which
has performance and driver compatibility benefits for the host

I think the benchmarks differ but show KVM having a bit of an edge. I
prefer the manageability of Xen. KVM is currently better at hot plug.

Neither of those aspects are particularly important to me, I just
need the guests to be stable and work properly with GPU passthrough,
and survive a reboot every week or two when Windows decides it needs
a reboot to apply the patches, without having to reboot the host.

It's a bit of a mess how Xen has migrated from xm to xl without
maintaining all the functionality, never mind in a solid manner.

I know what you mean. It seems to have been done without due

In fairness, it has been demonstrated many times that PCIe speed is
not particularly relevant for gaming loads. For compute loads that
are heavily reliant on shipping data to/from the GPU it will make
a difference, but if you your compute vs. I/O ratio is so low
the performance will be pretty horrible anyway.

This is true to some extent. My testing seems to show the difference
between PCI-e 1.x 16x and 8x is minimal, that the difference between
8x and 4x is (sadly) noticeable but not catastrophic, and then below
that things become a bit treacle like - although a fast card at 1x may
fare a lot better than a slow card at 8x.

Are you still running a PCIe 1.1 system? Or are you using a GPU that
is PCIe 1.1? IIRC most PCIe motherboards and devices since circa 2008
are PCIe 2.0+.


Xen-users mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.