[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-users] ATI VGA Passthrough / Xen 4.2 / Linux 3.8.10

On 05/09/2013 06:35 PM, Casey DeLorme wrote:
Thanks for posting the results Gordan, unfortunate that it isn't

        working as well as we hoped.

    I haven't given up _quite_ yet.

    I discovered yesterday that it _looks liks_ one of my PCIe slots is
    actually duff (two different GPUs both fail to detect properly in it
    but work fine in other slots).

    If it turns out to be a duff slot, there's no telling what else
    might be duff on the motherboard and how it might affect various
    things, even though several days of full load stability testing

    So some more bare-metal testing seems to be called for - right now I
    am not prepared to disregard the possibility that maybe I have a
    hardware issue somewhere that despite EDAC and ECC on everything,
    remains undetected and unreported in the logs.

I hope you manage to resolve it, though I feel the NF200 will be the
larger challenge.

I hope I'll resolve it, too, but right now I am not convinced that the NF200 is actually the cause of my problems. My gut feeling says that if I can get it working for 5 minutes at a time, something less fundamental than the NF200 PCIe routers are the cause of the problems.

           2) My motherboard's PCIe slots are behind NF200 PCIe bridges
        EVGA have decided in their infinite wisdom to put all 7 PCIe slots
        behind NF200s, none are directly attached to the Intel NB).

          I'm so sorry :P. NF200 has probably caused a lot of xen
        tinkerers to
          utter a few dozen cuss words a piece.

          I can believe that. What is the solution, though?

          The thing that drives me really nuts about the issues I'm seeing
        (which may or may not be specifically related to the NF200) is
        that it
        is so intermittent. It works well enough to boot up and work with a
        gaming type load for a few minutes. Then something happens that
        the VGA card to require a reset, and it all falls apart.

        My solution was to buy another motherboard, I had no luck at all
        passing the devices behind the NF200, and similar to your situation
        all but one PCIe slot on that board was behind that bridge.

    Did you not manage to get it working at all? Or was it just
    intermittent like in my case? I can typically get about 5 minutes of
    gaming out of my ATI card before it all goes wrong.

    Ironically, I was thinking about an Asus Sabertooth with an 8-core AMD,
    but opted to go for broke and get a couple of 6-core Xeons and an
    EVGA SR-2. It turns out, a solution that is 4x more expensive isn't
    actually better... :(

I was unable to get it working at all.  The NF200 simply threw errors
that 100% prevented me from passing the device.  I think it was missing
a number of specific features required for passthrough, and I vaguely
remember running lspci -vvv to verify what was missing.  Perhaps not all
NF200's are created equal?

The only logged issue I had with the NF200s was the lack of ACS, which can be disabled as I mentioned on this thread (at least if you are using the xm stack). After I disabled that PCI passthrough has been working OK. It's just VGA passthrough BSOD-ing after some minutes that is causing me problems.

           What about with PCIe devices behind NF200 bridges? I know the
        don't support PCI ACS, but that is a security feature (which I have
        disabled enforcement of to get this far), and AFAIK shouldn't
        affect the basic PCI passthrough capability.

          Question: how'd you disable ACS?  I think it may be causing me


          (pci-passthrough-strict-check no)
          (pci-dev-assign-strict-check no)

          in /etc/xen/xend-config.sxp

          If it was causing you issues, however, I'd expect you to find
        in logs pointing at it.

        As I understand the xend-config.sxp [1] is for the xm toolstack and
        deprecated Xend service.

    xm toolstack and xend are what I am using. I have read reports of issues
    with VGA passthrough using the xl stack so I didn't even attempt to
    use it.

The xm toolstack was deprecated in version 4.1.  I read that it had not
been updated in months due to a lack of maintainers.

I heard that xl is still feature-incomplete and experimental, and problematic with VGA passthrough.

I did try xm back
when I started, the passthrough worked but had the same problems I had
when I began testing xl.  I have been using xl since then.  My logic was
simply "why become dependent on a tool that is no-longer maintained and
may be removed from the next release?"

I'm not wedded to any particular tool stack, I'm happy to use whatever works. But since libvirt and virt-manager are still using xm, and since I have seen recent reports of xl being problematic for VGA passthrough as well as there being no apparent way to disable ACS requirements with the xl stack, that rules it out for me completely at the moment.

Does anyone know whether the xm toolstack been modified since 4.1 to
accommodate changes with Xen 4.2?  If it has not, it might be worth
considering xl.

Does anyone know how to disable the ACS bridge requirement with the xl stack?

        Perhaps I am confused, or things changed while I wasn't looking, but
        for me enabling Xend breaks the xl toolstack.  My understanding
        is it
        was for the xm toolstack only and deprecated with 4.2.  Any chance
        you can share how you configured it to work?  Apparently it is
        required to get libvirt working, which I also did not know was
        compatible with Xen 4.2.

    It is possible I'm the one doing it wrong. I'm on EL6, and using
    virt-manager (at least for things it is willing to do), and that
    defaults to the xm stack and xend.

    For what it's worth, it works for the most part - apart from VGA
    passthrough crashing within 5 minutes of gaming.

If you are using xm then it makes sense, as libvirt seems to require
xm/xend to be loaded in order to function.

There are more upgrade notes
xend now, so that is new to me.  According to the Xen Man Pages the
xend-config.sxp file doesn't have the flags you added; can you link to
resources that mentioned them?  I have not seen xl equivalents for your
xend configuration, so I guess xm does have some features xl does not still.

This mentions it, among others:

Google for
xen pci-passthrough-strict-check pci-dev-assign-strict-check

and you should find some relevant things easily enough.


Xen-users mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.