Xen project Mailing List

Gustav,

I am not sure I understand exactly where you are getting held up at, but were you able to get any video out of the card during passthrough or did it immediately go to Code 43? ÂAlso, can you share your lspci -v && lspci --tv output with us?

Toolstack is still a matter of preference, I use xl now so I don't have to worry about the eventual transition, but it might be worth trying the xm toolstack. ÂIt's clear that David has had much more luck using it than xl.

However, I would update your DomU configuration first and give it another try without David's script. ÂHis script is meant to pass the devices at run-time, which is helpful if you have two DomU's and only two graphics cards (eg. your discrete & integrated), but if you are leaving the integrated attached to Dom0 (or using ssh) that's not really a problem.

One of the biggest problems I have had with passthrough is the degraded state. ÂNot everyone has had the same experiences, but for me whenever the card is initialized (eg. Dom0 boot without passthrough, any DomU boot or reboot) subsequent initializations cause the degraded state. ÂFor me while in that degraded state installing and removing drivers has never succeeded. ÂIt also is unreversible in the Windows 7 VM's I tried. ÂSo my approach is generally to setup the system over VNC or SDL first, omitting all pci devices, then creating a dd backup of the logical volume to restore in the event of a problem during graphics install.

I won't be home for few more hours, but let me know if you want me to send the documentation I have currently. ÂMy current system is not entirely error-free (degradation and RAM limit exist and it has behaved a bit odd during some tests), but I have a working Windows DomU /w passthrough.

~Casey

On Thu, Jul 11, 2013 at 4:53 PM, Gustav Sorenson <gu.sorenson@xxxxxxxxx> wrote:

Hello everyone,

thank you for your replies, Gordan and Casey.

@Gordan:
What are XSA patches, and should I give them a try?

In my setup, I won't have the need to run more than two displays at a time, so I'd be happy if I was able to do with just the GPU integrated in the APU plus one dedicated GPU. Also, I know that with the integrated GPU, I won't be able to play newer games at maximum settings in high resolutions, but that's fine for me.

Reducing the DomU RAM to 1024 MB didn't solve my problem.

@Casey:
Your assumption is correct: I'm using the xl toolstack, sorry for not mentioning that earlier.

Your questions regarding my settings in the DomU configuration all share the same answer: Because David, who had succeeded in doing what I plan to do, had those settings in his config, and I copied them over, only changing things in cases where I was sure I knew the implications.

I've read about the performance degradation issues with AMD. While it's a pity that this issue exists, it would be one I would be able to accept, reading that nVidia only seems to work better when using Quadro cards which are out of my budget range. Since I haven't done any hardware hacking before and therefore wouldn't feel confident voiding my warranty, hacking a GeForce to get a cheaper Quadro isn't really an option either.

Thank you both for your help.

However, I've run out of things I could try to get passthrough of the integrated GPU to work, so I'd be very grateful for any further input.

Should this be a scenario that ought to be supported already, of course I'm willing to do the best I can to help debug this problem; please just let me know how.

On Thu, Jul 11, 2013 at 4:51 PM, Casey DeLorme <cdelorme@xxxxxxxxx> wrote:

Hi Gustav,

First, some more references:

- [Official Kernel Docs](http://wiki.xen.org/wiki/Mainline_Linux_Kernel_Configs)

- [Official Compiling Xen Docs](http://wiki.xen.org/wiki/Compiling_Xen_From_Source)

While they do not include steps for patching they may supplement David's guide.

---

Second, I reviewed your DomU config, under the assumption that you are using the `xl` toolstack and have some questions:

- What is `firmware_override`?

- What is `xen_extended_power_mgmt`?
- What is `monitor`?
- What is `audio`?

- Why set `extra` for an HVM?
- Why override the device model paths?
- Why use `shadow_memory` when you have HAP?

- Why set `serial` for a Windows VM?

- Why set `on_poweroff` to its default value?
- Why set `on_reboot` to its default value?
- Why set `sdl` to its default value?

- Why set `acpi` to its default value?
- Why set `apic` to its default value?

According to the Xen Man Pages, the xl configuration does not have a `firmware_override`, `monitor`, `audio` or a `xen_extended_power_mgmt` option. ÂThe `firmware_override` option is for PV guests, aimed at linux (nouveau being a linux nvidia driver?). ÂYou can omit options instead of setting them to their default value.

AFAIK the 2GB RAM limit applies to qemu upstream and you have set your device model to traditional, so you _should not_ be subject to this limitation (I can't speak for all cases, but I have never had the memory problem with traditional).

Integrated isn't great for gaming, but should be perfectly fine for Dom0 to use when you pass the discrete cards to VM's.

As Gordon has mentioned, AMD cards do work without patches to Xen source, but not as primary and they also experience performance degredation. ÂSince moving to upstream qemu I have encountered the RAM limit and the ejection trick no-longer works the same. ÂWhen I eject it does not automatically reinitialize, instead it disappears from the DomU (and puts errors in my qemu logs), but if I eject before shutting down or restarting I can shutdown from VNC or SDL and the card works without the degredation when started back up. ÂOtherwise the whole machine has to be restarted to resolve performance problems.

I am still finding 4.3 to be buggy, so I haven't nailed down my own steps to share, but I hope this information is helpful.

~Casey

On Thu, Jul 11, 2013 at 9:49 AM, Gordan Bobic <gordan@xxxxxxxxxx> wrote:

I wasn't aware that there was a restriction
regarding the DomU RAM. However, it just so
happens that I always tried with "memory=2048".

So far so good.

I assume this satisfies the <= 2GB RAM
requirement, although I'm not sure since I
don't know whether you refer to "base-10 G"
or "base-2 G". :)

Base 2. Perhaps I should have said 2GiB. :)

Furthermore, I don't know whether the setting
"shadow_memory = 512" counts towards that
limit,

What does that do?

or whether the 1024MB that has been assigned
to the integrated GPU in BIOS does.

Your GPU has a 1GB BAR? Really? I don't think
I've ever seen GPU BARs > 256MB. That could
potentially shrink the amount of RAM you
can reliably give the domU. Try with
memory=1024

Note: This should be fixed in 4.3, but I haven't
tried it yet.

Yes, I have tried without any first party
patches, as I had written in my first mail in
this thread. However, I haven't tried
"unpatched" xen with kernel 3.8.13 yet - should I?

I got this working with 4.2.2 + XSA patches
(mainly because that is what is in the RPMs I
use, I'm on EL6).

Unfortunately, I don't have a spare dedicated
graphics card I could add right now. I will try
to get one (is it important whether it's PCI or
PCIe, nVidia or ATI, or some specific model?)
and report back (if) when I get hold of one.

There is no particular requirement but it will
make it simpler if you have a dom0 card that
doesn't use the same drivers as the domU cards.
Otherwise, if you have xen-pciback as a module
you have to do some extra configuration/scripting
to ensure that upon loading the GPU driver, the
xen-pciback gets assigned the GPUs you don't want
the dom0 driver to handle. My setup is like this
at the moment and it works fine, but to make my
life easier I am going to switch to using
an old ATI 4850 (best I can get ATI-wise that is
single-slot and has two DL-DVI ports) for dom0,
and a pair of GeForce GTX480 modified into
Quadro 6000s for two separate domUs. I'm
currently using a GeForce 8800GT for dom0,
which is what I'll be replacing with a HD4850
in the near future (i.e. when I get around to
acquiring one).

However, my primary goal was to build a
computer that would serve two independent VMs
with 3D acceleration each, e.g. so that two
people could play games at the same time.

This is _precisely_ my use case, with the
extra requirement of also being my primary
workstation at the same time (without that
purpose having to be interrupted for gaming
purposes).

My plan was to buy the hardware as I have it
right now, test whether I could make xen
pass-through the one (integrated) GPU I have
right now,

Oh, is THIS what you were referring to by
1GiB for the GPU? That's not the same as BARs,
disregard what I said above.

and then buy a not-too-expensive
but still quite powerful dedicated GPU.
I would be much happier if I didn't have
to buy and install a third GPU, although
I realize me wishing that doesn't necessarily
make it so. ;)

My general experience is that integrated GPUs
aren't really up to gaming requirements in the
majority of cases, so my advice would be to get
a pair of different GPUs for VGA passthrough
and stick with the integrated one for dom0.

My experience with ATI for VGA passthrough has
been somewhat poor, but provided you avoid
PCI memory stomps and weird IRQ clash issues,
the experience with Nvidia Quadros has been
very positive. GeForce cards won't work
until/unless you modify them into equivalent
Quadros. See:
http://www.altechnative.net/2013/06/23/nvidia-cards-geforce-quadro-and-geforce-modified-into-a-quadro-for-virtualized-gaming/

One thing that ATI users seem to be experiencing
is progressive graphics slow-down after reboots
in domUs with VGA passthrough, which requires
host reboot to fix. I have not experienced this
with my Quadrified GeForce cards.

It took a lot of effort to get this working and
work around all the issues, though. Which slots
you have the hardware in makes a difference, as
does the nature and number of PCIe bridges
involved, as well as the combination of hardware
you are passing through. ACS support on the
PCIe bridges may also help ensure that a bug
causing a PCI memory stomp doesn't crash dom0,
although I have managed to get things working
stably and reliably without it.

Gordan

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx

http://lists.xen.org/xen-users

Re: [Xen-users] VGA Passthrough of AMD HD8670D IGP to HVM Win7 results in "Code 43"