[Xen-users] AMD cards - VGA passthrough no more possible with newer Xen / kernel releases

["H. Sieger" <powerhouse.linux@xxxxxxxxx>]

I've recently tested both my AMD HD 7770 and my AMD HD 6450 video cards for VGA passthrough and had a hell of a time getting it working. Here the results of my tests:

1. Kernel 3.11.0-19-generic with Xen 4.3.0 and xm toolstack

dom0 OS: Linux Mint 16 (based on Ubuntu 13.10), kernel info see above

domU: Windows 7 Pro 64 bit

GPU for VGA passthrough: Sapphire HD 6450 (AMD), secondary passthrough

Method: xen-pciback module loaded from initramfs:

echo "xen-pciback passthrough=1 hide=(02:00.0)(02:00.1)" >> /etc/initramfs-tools/modules

Result with xm create /etc/xen/win7.cfg:Error: (22, 'Invalid argument')

The same win7.cfg file with a Nvidia Quadro 2000 works perfectly. Other Xen users have reported the same problem, see here.

2. Kernel 3.11.0-19-generic with Xen 4.3.0 and xl toolstack

dom0 OS: Linux Mint 16 (based on Ubuntu 13.10), kernel info see above

domU: Windows 7 Pro 64 bit

GPU for VGA passthrough: Sapphire HD 6450 (AMD), secondary passthrough

Method: xen-pciback module loaded from initramfs:

echo "xen-pciback passthrough=1 hide=(02:00.0)(02:00.1)" >> /etc/initramfs-tools/modules

I was able to start the Windows installation in domU but after a domU reboot it wouldn't start:


> xl create /etc/xen/win7.cfg
Parsing config from /etc/xen/win7.cfg
WARNING: specifying "tsc_mode" as an integer is deprecated. Please use the named parameter variant. e.g. tsc_mode="default"
WARNING: ignoring "kernel" directive for HVM guest. Use "firmware_override" instead if you really want a non-default firmware
xc: info: VIRTUAL MEMORY ARRANGEMENT:
  Loader:        0000000000100000->00000000001a69a4
  Modules:       0000000000000000->0000000000000000
  TOTAL:         0000000000000000->00000000ff800000
  ENTRY ADDRESS: 0000000000100608
xc: info: PHYSICAL MEMORY ALLOCATION:
  4KB PAGES: 0x0000000000000200
  2MB PAGES: 0x00000000000003fb
  1GB PAGES: 0x0000000000000002
libxl: error: libxl_device.c:894:device_backend_callback: unable to add device with path /local/domain/0/backend/vbd/3/768
libxl: error: libxl_create.c:932:domcreate_launch_dm: unable to add disk devices
libxl: error: libxl_dm.c:1300:libxl__destroy_device_model: could not find device-model's pid for dom 3
libxl: error: libxl.c:1409:libxl__destroy_domid: libxl__destroy_device_model failed for 3
libxl: error: libxl_device.c:894:device_backend_callback: unable to remove device with path /local/domain/0/backend/vbd/3/768
libxl: error: libxl.c:1445:devices_destroy_cb: libxl__devices_destroy failed for 3


I've been trying different options with the win7.cfg file in conjunction with xl. I had to reboot (hard reset) the PC each time I shut down the Windows domU (what a pain in the neck).

In short, the xl toolstack in Xen 4.3.0 is totally useless with VGA passthrough and at least 2 AMD cards: HD 6450 and HD 7770.

I tried resetting the GPU in the domU which then prevented the domU from switching to the secondary graphics display (the HD 7770 or HD 6450).

3. Kernel 3.2.0-23-generic with Xen 4.1.2-2ubuntu2 and xm toolstack

dom0 OS: Linux Mint 13 (based on Ubuntu 12.04), kernel info see above

domU: Windows 7 Pro 64 bit

GPU for VGA passthrough: Sapphire HD 6450 (AMD), secondary passthrough

Method: xen-pciback module loaded from initramfs.

The same AMD cards work just perfect with the xm toolstack, kernel 3.2.0-23-generic and Xen 4.1.2-2ubuntu2 on a Linux Mint 13 dom0, after downgrading both the kernel and Xen to the earliest release. Somewhere down the kernel/Xen upgrade path things went wrong. See here and here.

Conclusion and questions:

With xm deprecated and xl not up to snuff, what's the future of VGA passthrough under Xen?

Does anyone here have a solution? It can't be that AMD users have to use ancient kernel and Xen releases with a deprecated xm toolstack to enjoy VGA passthrough. Or does Xen 4.4 answer these issues and work with xl and AMD cards?

P.S.: I would file bug reports but it seems pointless, there are just too many things going wrong. The error 22 with xm should have been solved long ago, and yes, I checked the python code and the bug fix has been applied in Xen 4.3.0, yet there is again the infamous error 22 message.

As to the domU shutdown/reboot issue with xl it has been reported many times. The only thing I haven't read yet is that someone is trying to fix this. Or is there actually a fix or release that works?

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users