Just wanting to check the basics - have you also make sure you've installed the latest BIOS update for your motherboard? Sometimes these updates contain fixes fror issues such as bad table entries.
David,
Yes, I have the latest BIOS update, F10a, for my GA-990FXA-UD3 ver1.1. PCI passthrough works fine for USB controllers but when I try to passthrough a secondary VGA card I see crash logs as copied in the attachment. If you, or any others on the list, have ideas on how to work through this issue I would be very interested in hearing them.
Thanks,
Eric
I've been looking through the log file and my first instinct is that you're experiencing a driver issue on dom0, which is causing the issue (basically a driver on dom0 has already claimed the device and doesn't like that you're trying to take it away). To validate that, I'd need some more information; You mention that this is a dual GPU system; are both GPUs ATI/AMD/Radeon? How are you making the second GPU available to be passed through (via xl commands? modprobe options? passed kernel options?)
David,
This box does have two GPUs:
[root@xen ~]# lspci | grep VGA
01:00.0 VGA compatible controller: NVIDIA Corporation GT218 [GeForce 210] (rev a2)
04:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Juniper PRO [Radeon HD 5750]
I have the NVIDIA for use with Dom0 and would like to pass the Radeon HD 5750 to a VM running Win7. To assign the 5750 to pciback, I have tried several methods as shown on the Xen PCI Passthrough Wiki at http://wiki.xen.org/wiki/Xen_PCI_Passthrough. Dynamic Assignment to pciback with xl and sysfs both result in similar errors as shown in my previous email . To prevent the Radeon driver from loading at boot time, I tried Static Assignment but it fails to stop the Radeon driver from loading. Here is what I think is the relevant configuration for Static Assignment, maybe I have not followed the Wiki accurately:
[root@xen ~]# lspci –k
04:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Juniper PRO [Radeon HD 5750]
Subsystem: Hightech Information System Ltd. Device 2287
Kernel driver in use: radeon
[root@xen ~]# cat /etc/modprobe.d/radeon.conf
install radeon /sbin/modprobe xen-pciback ; /sbin/modprobe --first-time --ignore-install radeon
options xen-pciback hide=(0000:04:00.0)
[root@xen ~]# lsinitrd | grep pciback
drwxr-xr-x 2 root root 0 Dec 28 10:16 usr/lib/modules/3.12.5-302.fc20.x86_64/kernel/drivers/xen/xen-pciback
-rw-r--r-- 1 root root 106151 Dec 17 14:03 usr/lib/modules/3.12.5-302.fc20.x86_64/kernel/drivers/xen/xen-pciback/xen-pciback.ko
After a reboot xen-pciback is not loaded and the 5750 is still using the radeon driver.
Thanks,
Eric
To try and get this initially working, i'd go ahead and just blacklist the radeon driver, since your primary GPU is a NVIDIA card. Also, I'd check but you may find that your AMD card actually has two PCI entries, one for graphics and one for HDMI audio (most of the ones that I've played with have) - probably something like 04:00.1 or 04:00.2; if you do, i'd pass that one through as well. You can then use 'xl pci-assignable-add' to add the one or two entries, once the radeon driver isn't bound to it. That should at least yet us to where you are able to boot into the dual environment; I've had a similar setup in the past, an NVIDIA GeForce 550Ti as my primary, AMD Radeon 6770 passed through to a Windows 7 HVM domain, on an ASUS Sabertooth 990FX board ( AMD Phenom II x4 965, or an AMD FX 8350 cpu)
David,
I figured out what was wrong with my Static Assignment configuration. When I initially built initrd to include the xen-pciback module, I had a typo in my /etc/modprobe.d/radeon.conf file, the same typo as is on the Wiki – highlighted in yellow below. After correcting the typo in the configuration file I did not rebuild initrd, which is required, at least on Fedora 20. It was easy to see why a rebuild of initrd is required once I played around with lsinitrd.
[root@xen grub.d]# lsinitrd | grep radeon.conf
-rwxr-xr-x 1 root root 178 Jan 3 16:06 etc/modprobe.d/radeon.conf
[root@xen grub.d]#
[root@xen grub.d]# cat /etc/modprobe.d/radeon.conf
install radeon /sbin/modprobe xen-pciback ; /sbin/modprobe --first-time --ignore-install radeon
options xen-pciback hide=(0000:04:00.0)(0000:04:00.1)
[root@xen grub.d]#
I rebuilt initrd with dracut –f --add-drivers xen_pciback and now the HD 5750 is cleanly assigned to pciback at boot.
[root@xen grub.d]# lspci –k
04:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Juniper PRO [Radeon HD 5750]
Subsystem: Hightech Information System Ltd. Device 2287
Kernel driver in use: pciback
04:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Juniper HDMI Audio [Radeon HD 5700 Series]
Subsystem: Hightech Information System Ltd. Device aa58
Kernel driver in use: pciback
The good news is that pci passthrough of the HD 5750 to a Win7 VM is working. The bad news is that it will only work for the first boot of the VM. If the VM is rebooted, the guest appears to be unable to complete the video initialization. I see a mouse cursor over a black screen, the cursor moves when the mouse is moved, but the screen stays black. It is possible to remote desktop to the VM and everything appears to work as expected. A reboot of the whole box will clear this condition and pci passthrough of the HD 5750 will work one more time.
Thanks,
Eric
