Re: [Xen-users] ATI VGA Passthrough / Xen 4.2 / Linux 3.8.10

[Gordan Bobic <gordan@xxxxxxxxxx>]

As promised, here is an initial report on how the Quadro 2000 experiment 
panned out.
In short - if anything stability is a little _worse_, with all the same 
issues that require a host reboot after a guest crash.
So, as a list, both ATI and Nvidia Quadro suffer the following issues:

1) Random guest graphics corruption (horizontal lines) after 
almost-but-not-quite-proper crash.
2) Same BSODs reporting timeout while attempting to reset a device.

3) Similar but seemingly worse graphics stability. With the ATI card I 
managed about 5 minutes in Borderlands 2 before it crashed. With the 
Quadro I seem to be managing about 2 minutes before it tries to reset 
itself and falls flat on it's face.
The one advantage the quadro has is that once it has crashed, the driver 
disables the card, so the guest doesn't repeately BSOD until you reboot 
the host. Instead it gives you the login screen on the primary (VNC) 
display output, so you can still get into the guest easily and do any 
required maintenance. Futile, but it's an improvement on just BSOD-ing.
The only two things that come to mind as possible causes are:
1) I'm on a dual socket system (dual Xeon X5650)
2) My motherboard's PCIe slots are behind NF200 PCIe bridges (yes, EVGA 
have decided in their infinite wisdom to put all 7 PCIe slots behind 
NF200s, none are directly attached to the Intel NB).
Has anyone managed to successfully run VGA passthrough on a dual socket 
system?
What about with PCIe devices behind NF200 bridges? I know the NF200s 
don't support PCI ACS, but that is a security feature (which I have 
disabled enforcement of to get this far), and AFAIK shouldn't actually 
affect the basic PCI passthrough capability.
All in all - rather deflating. I was really hoping that I wouldn't have 
to be stuck with rebooting between OS-es for another hardware generation.
Gordan

On 05/05/2013 11:19 PM, Gordan Bobic wrote:
> OK, I think I have finally managed to get things to what appears to be
> the average state of unreliability of ATI VGA passthrough (i.e. seems to
> work _most_ of the time after a fresh reboot, but becomes much more hit
> and miss after a VM reboot or two).
>
> I'm probably going to regret saying this when I find that over the next
> week I cannot get it to start up even once, fresh reboot or not, but
> what seems to have made a difference is disabling irq balancing (I'm on
> a dual X5650 machine, 2 CPUs, 6 cores / 12 threads each).
>
> So if you are having similarly difficult time, you might want to try the
> noirqbalance dom0 kernel boot parameter and disable the irqbalance service.
>
> Disclaimer: This is largely based on a gut feeling after a few hours
> testing, I certainly don't think it's definitive, but statistically it
> seems to help.
>
> I have a Quadro 2000 inbound, so I will test with that when it arrives
> next week. The optimist in me very much hoping it will "just work". The
> realist suspects that would be way too easy. Will know for sure one way
> or the other in a few days' time.
>
> Gordan
>
>
> On 05/05/2013 04:57 PM, Gordan Bobic wrote:
> > On 05/05/2013 04:42 PM, Gordan Bobic wrote:
> > > This problem continues to drive me nuts - not by it flat out not
> > > working, but by working _intermittently_.
> > >
> > > For the past week, I had not managed to get ATI VGA passthrough to boot
> > > up once (BSOD every time). I was tweaking some boot parameters, and at
> > > one point it not only booted up without BSOD-ing, it actually managed
> > > full screen 3D applications, and completed a full GPU benchmark pass of
> > > Crysis!
> > >
> > > So just to make sure, I did a full shutdown and cold-booted the machine
> > > again - BSOD after BSOD after BSOD.
> > >
> > > Rebooted it again, and now it works again, including full screen 3D
> > > switching.
> > >
> > > One thing I have established is that pci=resource_alignment=<id>;<id>
> > > kernel boot parameter makes the machine not boot at all. It looks like
> > > it wipes out the NIC by realigning things, and I need the NIC to work
> > > because the machine runs on NFS root (and the VM disk is an iSCSI
> > > share).
> > >
> > > Has anybody got any suggestions on how I might debug this any further?
> > > It's really quite annoying having this _almost_ working. It also looks
> > > like if it works once, it will continue working upon guest restarts.
> > Another observation - when it boots up, and I "eject" the ATI card from
> > the guest, all that does it switch the guest display briefly to VNC
> > primary, followed by slightly corrupted output switching back to the ATI
> > card. And it is still responsive after that, I can execute a normal
> > shutdown, don't have to do it blind.

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users