[Xen-users] Win8.1 64-bit VGA passthrough - Code 12s and host crashes

[Stephen Oberholtzer <stevie@xxxxxxxxx>]

All,

(I couldn't find anything online on what information I should provide, so I'm going to present a minimal amount so as to not fill everybody's mailbox with useless junk.)

I built a machine with the intention of running Windows in a VM with VGA passthrough to a PCI card.

I mostly followed instructions laid out here: http://forums.linuxmint.com/viewtopic.php?f=42&t=112013

Dom0 is Debian Jessie (amd64).

'xl info' output:
hostÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ : zen
releaseÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ : 3.16-3-amd64
versionÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ : #1 SMP Debian 3.16.5-1 (2014-10-10)
machineÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ : x86_64
nr_cpusÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ : 8
max_cpu_idÂÂÂÂÂÂÂÂÂÂÂÂ : 7
nr_nodesÂÂÂÂÂÂÂÂÂÂÂÂÂÂ : 1
cores_per_socketÂÂÂÂÂÂ : 4
threads_per_coreÂÂÂÂÂÂ : 2
cpu_mhzÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ : 3199
hw_capsÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ : bfebfbff:2c100800:00000000:00007f00:77fafbff:00000000:00000021:00002fbb
virt_capsÂÂÂÂÂÂÂÂÂÂÂÂÂ : hvm hvm_directio
total_memoryÂÂÂÂÂÂÂÂÂÂ : 32445
free_memoryÂÂÂÂÂÂÂÂÂÂÂ : 23829
sharing_freed_memoryÂÂ : 0
sharing_used_memoryÂÂÂ : 0
outstanding_claimsÂÂÂÂ : 0
free_cpusÂÂÂÂÂÂÂÂÂÂÂÂÂ : 0
xen_majorÂÂÂÂÂÂÂÂÂÂÂÂÂ : 4
xen_minorÂÂÂÂÂÂÂÂÂÂÂÂÂ : 4
xen_extraÂÂÂÂÂÂÂÂÂÂÂÂÂ : .1
xen_versionÂÂÂÂÂÂÂÂÂÂÂ : 4.4.1
xen_capsÂÂÂÂÂÂÂÂÂÂÂÂÂÂ : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
xen_schedulerÂÂÂÂÂÂÂÂÂ : credit
xen_pagesizeÂÂÂÂÂÂÂÂÂÂ : 4096
platform_paramsÂÂÂÂÂÂÂ : virt_start=0xffff800000000000
xen_changesetÂÂÂÂÂÂÂÂÂ :
xen_commandlineÂÂÂÂÂÂÂ : placeholder iommu=1 dom0_mem=8192M no-real-mode edd=off
cc_compilerÂÂÂÂÂÂÂÂÂÂÂ : gcc (Debian 4.9.1-17) 4.9.1
cc_compile_byÂÂÂÂÂÂÂÂÂ : waldi
cc_compile_domainÂÂÂÂÂ : debian.org
cc_compile_dateÂÂÂÂÂÂÂ : Fri Oct 17 14:29:43 UTC 2014
xend_config_formatÂÂÂÂ : 4

The Windows VM is Windows 8.1, 64-bit.

These are the devices passed through:

00:14.0 USB controller: Intel Corporation 9 Series Chipset Family USB xHCI Controller (prog-if 30 [XHCI])
ÂÂÂÂÂÂÂ Subsystem: ASRock Incorporation Device 8cb1
ÂÂÂÂÂÂÂ Flags: medium devsel, IRQ 16
ÂÂÂÂÂÂÂ Memory at efd20000 (64-bit, non-prefetchable) [disabled] [size=64K]
ÂÂÂÂÂÂÂ Capabilities: [70] Power Management version 2
ÂÂÂÂÂÂÂ Capabilities: [80] MSI: Enable- Count=1/8 Maskable- 64bit+
ÂÂÂÂÂÂÂ Kernel driver in use: pciback

01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Oland XT [Radeon HD 8670 / R7 250] (prog-if 00 [VGA controller])
ÂÂÂÂÂÂÂ Subsystem: XFX Pine Group Inc. Device 7250
ÂÂÂÂÂÂÂ Flags: fast devsel, IRQ 16
ÂÂÂÂÂÂÂ Memory at d0000000 (64-bit, prefetchable) [disabled] [size=256M]
ÂÂÂÂÂÂÂ Memory at efc00000 (64-bit, non-prefetchable) [disabled] [size=256K]
ÂÂÂÂÂÂÂ I/O ports at e000 [disabled] [size=256]
ÂÂÂÂÂÂÂ Expansion ROM at efc40000 [disabled] [size=128K]
ÂÂÂÂÂÂÂ Capabilities: [48] Vendor Specific Information: Len=08 <?>
ÂÂÂÂÂÂÂ Capabilities: [50] Power Management version 3
ÂÂÂÂÂÂÂ Capabilities: [58] Express Legacy Endpoint, MSI 00
ÂÂÂÂÂÂÂ Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
ÂÂÂÂÂÂÂ Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
ÂÂÂÂÂÂÂ Capabilities: [150] Advanced Error Reporting
ÂÂÂÂÂÂÂ Capabilities: [270] #19
ÂÂÂÂÂÂÂ Kernel driver in use: pciback

01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde/Pitcairn HDMI Audio [Radeon HD 7700/7800 Series]
ÂÂÂÂÂÂÂ Subsystem: XFX Pine Group Inc. Device aab0
ÂÂÂÂÂÂÂ Flags: fast devsel, IRQ 17
ÂÂÂÂÂÂÂ Memory at efc60000 (64-bit, non-prefetchable) [disabled] [size=16K]
ÂÂÂÂÂÂÂ Capabilities: [48] Vendor Specific Information: Len=08 <?>
ÂÂÂÂÂÂÂ Capabilities: [50] Power Management version 3
ÂÂÂÂÂÂÂ Capabilities: [58] Express Legacy Endpoint, MSI 00
ÂÂÂÂÂÂÂ Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
ÂÂÂÂÂÂÂ Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
ÂÂÂÂÂÂÂ Capabilities: [150] Advanced Error Reporting
ÂÂÂÂÂÂÂ Kernel driver in use: pciback

I have the USB ports successfully passed through to the VM; keyboard and mouse, USB hard drive, and when I plugged in my phone to charge it, it was recognized as well. However, Windows reports an error for the video card, "Code 12" something about resources.

I searched and found anecdotal evidence of a bug when the guest has >2GB allocated (I had given it 8). I would love to know more information about this bug, but I can't find an actual bug filed. If someone has more information about it, I would greatly appreciate hearing it.

In any case, I dropped the memory limit on the guest to 2GB and now it crashes at startup, rebooting into recovery mode. Whenever I turn on any of the debugging options (boot logging, debugging, disable automatic restart on failure) the entire host stops responding.

I searched more about this and the only reference I found that matched up with what I'm doing is here: http://lists.xenproject.org/archives/html/xen-users/2013-10/msg00042.html

It suggests that some motherboards have screwed up chipsets that bypass the IOMMU.

In conclusion, I have four questions:

(1) Does anyone have any more information on a PCI-passthrough bug with guests that have >2GB RAM?

(2) How can I find out if my motherboard has a bad PCIe bridge chipset that bypasses the IOMMU?

(3) Is there a way to convince Xen and Dom0 to not occupy the first 2GB of physical RAM, and have Xen load DomU into that space?
If I can do that, and the host stops crashing, that seems like it would be a good indicator that the IOMMU is being bypassed.

(4) What else can I do to troubleshoot all this?

--

-- Stevie-O
Real programmers use COPY CON PROGRAM.EXE

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users