Re: [Xen-users] ATI VGA Passthrough / Xen 4.2 / Linux 3.8.10

[Andrew Bobulsky <rulerof@xxxxxxxxx>]

Hello Gordan,

A couple suggestions and questions for you!

On May 8, 2013, at 6:53 PM, Gordan Bobic <gordan@xxxxxxxxxx> wrote:

> The only two things that come to mind as possible causes are:
> 1) I'm on a dual socket system (dual Xeon X5650)

There's a thread on xen-devel right now.... Someone attempting iommu
with a dual socket Xeon system having trouble... Devices showing up
twice with different addresses.  It might not help at all, but it
could be worth a peek.

> 2) My motherboard's PCIe slots are behind NF200 PCIe bridges (yes, EVGA have 
> decided in their infinite wisdom to put all 7 PCIe slots behind NF200s, none 
> are directly attached to the Intel NB).

I'm so sorry :P. NF200 has probably caused a lot of xen tinkerers to
utter a few dozen cuss words a piece.

> What about with PCIe devices behind NF200 bridges? I know the NF200s don't 
> support PCI ACS, but that is a security feature (which I have disabled 
> enforcement of to get this far), and AFAIK shouldn't actually affect the 
> basic PCI passthrough capability.

Question: how'd you disable ACS?  I think it may be causing me some issues.

Have you tried passing the NF200 bridge itself, along with the devices
behind it?  If its possible, it may enable a "cleaner" interaction as
the pcie bus ahead of it should be more well behaved.

> All in all - rather deflating. I was really hoping that I wouldn't have to be 
> stuck with rebooting between OS-es for another hardware generation.

Also, are you doing VGA pass through, or just pcie pass through of a
device that happens to be a video card?  I'm guessing you mean the
latter, but if not, it's definitely the more compatible option :)

Cheers,
Andrew



> Gordan
>
> On 05/05/2013 11:19 PM, Gordan Bobic wrote:
>> OK, I think I have finally managed to get things to what appears to be
>> the average state of unreliability of ATI VGA passthrough (i.e. seems to
>> work _most_ of the time after a fresh reboot, but becomes much more hit
>> and miss after a VM reboot or two).
>>
>> I'm probably going to regret saying this when I find that over the next
>> week I cannot get it to start up even once, fresh reboot or not, but
>> what seems to have made a difference is disabling irq balancing (I'm on
>> a dual X5650 machine, 2 CPUs, 6 cores / 12 threads each).
>>
>> So if you are having similarly difficult time, you might want to try the
>> noirqbalance dom0 kernel boot parameter and disable the irqbalance service.
>>
>> Disclaimer: This is largely based on a gut feeling after a few hours
>> testing, I certainly don't think it's definitive, but statistically it
>> seems to help.
>>
>> I have a Quadro 2000 inbound, so I will test with that when it arrives
>> next week. The optimist in me very much hoping it will "just work". The
>> realist suspects that would be way too easy. Will know for sure one way
>> or the other in a few days' time.
>>
>> Gordan
>>
>>
>> On 05/05/2013 04:57 PM, Gordan Bobic wrote:
>>> On 05/05/2013 04:42 PM, Gordan Bobic wrote:
>>>> This problem continues to drive me nuts - not by it flat out not
>>>> working, but by working _intermittently_.
>>>>
>>>> For the past week, I had not managed to get ATI VGA passthrough to boot
>>>> up once (BSOD every time). I was tweaking some boot parameters, and at
>>>> one point it not only booted up without BSOD-ing, it actually managed
>>>> full screen 3D applications, and completed a full GPU benchmark pass of
>>>> Crysis!
>>>>
>>>> So just to make sure, I did a full shutdown and cold-booted the machine
>>>> again - BSOD after BSOD after BSOD.
>>>>
>>>> Rebooted it again, and now it works again, including full screen 3D
>>>> switching.
>>>>
>>>> One thing I have established is that pci=resource_alignment=<id>;<id>
>>>> kernel boot parameter makes the machine not boot at all. It looks like
>>>> it wipes out the NIC by realigning things, and I need the NIC to work
>>>> because the machine runs on NFS root (and the VM disk is an iSCSI
>>>> share).
>>>>
>>>> Has anybody got any suggestions on how I might debug this any further?
>>>> It's really quite annoying having this _almost_ working. It also looks
>>>> like if it works once, it will continue working upon guest restarts.
>>>
>>> Another observation - when it boots up, and I "eject" the ATI card from
>>> the guest, all that does it switch the guest display briefly to VNC
>>> primary, followed by slightly corrupted output switching back to the ATI
>>> card. And it is still responsive after that, I can execute a normal
>>> shutdown, don't have to do it blind.
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@xxxxxxxxxxxxx
> http://lists.xen.org/xen-users

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users