[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-users] ATI VGA Passthrough / Xen 4.2 / Linux 3.8.10

As promised, here is an initial report on how the Quadro 2000 experiment panned out.

In short - if anything stability is a little _worse_, with all the same issues that require a host reboot after a guest crash.

So, as a list, both ATI and Nvidia Quadro suffer the following issues:

1) Random guest graphics corruption (horizontal lines) after almost-but-not-quite-proper crash.

2) Same BSODs reporting timeout while attempting to reset a device.

3) Similar but seemingly worse graphics stability. With the ATI card I managed about 5 minutes in Borderlands 2 before it crashed. With the Quadro I seem to be managing about 2 minutes before it tries to reset itself and falls flat on it's face.

The one advantage the quadro has is that once it has crashed, the driver disables the card, so the guest doesn't repeately BSOD until you reboot the host. Instead it gives you the login screen on the primary (VNC) display output, so you can still get into the guest easily and do any required maintenance. Futile, but it's an improvement on just BSOD-ing.

The only two things that come to mind as possible causes are:
1) I'm on a dual socket system (dual Xeon X5650)
2) My motherboard's PCIe slots are behind NF200 PCIe bridges (yes, EVGA have decided in their infinite wisdom to put all 7 PCIe slots behind NF200s, none are directly attached to the Intel NB).

Has anyone managed to successfully run VGA passthrough on a dual socket system?

What about with PCIe devices behind NF200 bridges? I know the NF200s don't support PCI ACS, but that is a security feature (which I have disabled enforcement of to get this far), and AFAIK shouldn't actually affect the basic PCI passthrough capability.

All in all - rather deflating. I was really hoping that I wouldn't have to be stuck with rebooting between OS-es for another hardware generation.


On 05/05/2013 11:19 PM, Gordan Bobic wrote:
OK, I think I have finally managed to get things to what appears to be
the average state of unreliability of ATI VGA passthrough (i.e. seems to
work _most_ of the time after a fresh reboot, but becomes much more hit
and miss after a VM reboot or two).

I'm probably going to regret saying this when I find that over the next
week I cannot get it to start up even once, fresh reboot or not, but
what seems to have made a difference is disabling irq balancing (I'm on
a dual X5650 machine, 2 CPUs, 6 cores / 12 threads each).

So if you are having similarly difficult time, you might want to try the
noirqbalance dom0 kernel boot parameter and disable the irqbalance service.

Disclaimer: This is largely based on a gut feeling after a few hours
testing, I certainly don't think it's definitive, but statistically it
seems to help.

I have a Quadro 2000 inbound, so I will test with that when it arrives
next week. The optimist in me very much hoping it will "just work". The
realist suspects that would be way too easy. Will know for sure one way
or the other in a few days' time.


On 05/05/2013 04:57 PM, Gordan Bobic wrote:
On 05/05/2013 04:42 PM, Gordan Bobic wrote:
This problem continues to drive me nuts - not by it flat out not
working, but by working _intermittently_.

For the past week, I had not managed to get ATI VGA passthrough to boot
up once (BSOD every time). I was tweaking some boot parameters, and at
one point it not only booted up without BSOD-ing, it actually managed
full screen 3D applications, and completed a full GPU benchmark pass of

So just to make sure, I did a full shutdown and cold-booted the machine
again - BSOD after BSOD after BSOD.

Rebooted it again, and now it works again, including full screen 3D

One thing I have established is that pci=resource_alignment=<id>;<id>
kernel boot parameter makes the machine not boot at all. It looks like
it wipes out the NIC by realigning things, and I need the NIC to work
because the machine runs on NFS root (and the VM disk is an iSCSI

Has anybody got any suggestions on how I might debug this any further?
It's really quite annoying having this _almost_ working. It also looks
like if it works once, it will continue working upon guest restarts.

Another observation - when it boots up, and I "eject" the ATI card from
the guest, all that does it switch the guest display briefly to VNC
primary, followed by slightly corrupted output switching back to the ATI
card. And it is still responsive after that, I can execute a normal
shutdown, don't have to do it blind.

Xen-users mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.