[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-users] GPU passthrough with xen 4.2.1 on ubuntu 12.10



That was a good suggestion, I've just tried passing through a USB controller and that worked fine.  The driver installed correctly and I can use devices attached to the relevant ports.  The problem therefore seems to be to do with gpu passthrough specifically.  I'm just going to describe my system and some settings I have in the bios as this is starting to feel like a system specific problem.  The system is

Asus Z9PE-D8 WS motherboard (chipset is intel C602)
Dual Xeon 2620
64 GB non-ecc memory
Dual AMD radeon HD 7970 in slots 1 & 5 (it's the one in slot 5 I'm trying to pass through)

In the bios I have vt-d enabled.  I also have sub-feature settings for "Coherency Support" and "ATS Support" which are disabled by default.  I've tried all combinations of these enabled and disabled.  If I enable "ATS Support" I do however have to add the option "iommu=no-qinval,no-intremap" to the xen boot otherwise the system hangs as per this thread.  If the problem is to no with these options then I think I have a problem.

Has anybody got any thoughts?

Matt

On 11/03/2013 14:57, Peter Maloney wrote:
Did you try other devices, such as your onboard sound card, your onboard USB controllers, etc.? Maybe your board doesn't do passthrough properly.

Or did you try another PCIe port for the card? Maybe there are bugs where the addresses overlap with some other device, but in another slot it wouldn't happen.


On 2013-03-11 14:48, Matthew Dean wrote:
First of all, thank you for the suggestions.  I've now re-installed windows and I've tried driver versions 12.3, 12.8 and 13.1 all with and without the CCC as I read in various places that it can cause problems.  For 13.1 I never get anywhere close as per my previous messages.  For 12.3 and 12.8 I at least get a BSOD,

#######################################################################

A problem has been detected ...

Attempt to reset the display driver and recover from timeout failed.

If this is ...

Techinal information:

*** STOP: 0x00000116 (0xFFFFFA8002AAF010,0xFFFFF88003D95AE4,0x0000000000000000,0x0000000000000002)

*** atikmpag.sys - Address FFFFF88003D95AE4 base at FFFFF88003D8E000, Datestamp 4f597fca

#######################################################################

Does anybody have any further ideas?

Matt

On 07/03/2013 21:42, John Sherwood wrote:
(and I should probably mention that it has been my experience that that is the expected and correct behavior)

On Thu, Mar 7, 2013 at 1:41 PM, John Sherwood <jrs@xxxxxx> wrote:
Having the windows logo freeze in the VNC window has been my experience with GPU passthrough as well.


On Thu, Mar 7, 2013 at 1:38 PM, Andreas Falck <falck.andreas.lists@xxxxxxxxx> wrote:
Hi,

I believe that the Windows logo freezing in the vnc window is normal behaviour when secondary GPU passthrough is working as it should, since the display should be handed over to the GPU. At least that was the case before when I was passing through a HD6850 card as secondary to win7 using xl. But you should expect output on the GPU, of course.

Have you tried different versions of the catalyst driver? I think that some users have reported problems with the latest one (13-something).

Regards,
Andreas

2013/3/7 Matthew Dean <mcd40@xxxxxxxxx>
Hi all,

I've managed to get a windows 7 x64 hvm to boot and install and I'm now looking to pass through a HD 7970 as a secondary GPU.  in order to prepare the devices for passthrough I've used

xl pci-assignable-add 83:00.0
xl pci-assignable-add 83:00.1

Where I've identified the device pci ids using lspci.  This works fine and the devices appear when I call 'xl pci-assignable-list'. After adding them to the vm config (see below) I was able to get the vm to boot and the devices appeared in device manager.  Installing the ATI drivers however has meant that on further boots the  machine never gets past the windows logo in the VNC window and no output is generated from the ATI card.  It simply freezes there and the only way I can get back into windows is via safe mode, even after uninstalling the ATI software.

Has anybody else experienced similar issues or have any advice as to how to get this to work?  I'm a bit stumped and the wiki implies that this should 'just work'.

Thank you in advance for any help.

Matt

Here is my machine config

###########################################################

builder='hvm'
memory = 2048
name = 'Windows7Test'

vcpus=4
pae=1
acpi=1
apic=1

disk = [ 'file:/xen-guests/img/windows7Test.img,hda,w' ]
pci = [ '83:00.0' , '83:00.1' ]


on_reboot   = 'restart'
on_crash    = 'destroy'

boot='cd'

sdl=0
vnc=1
vncconsole=0
vncpasswd=''
vnclisten='172.24.226.84'

stdvga=0

serial='pty'
usbdevice='tablet'

gfx_passthru=0
pci_power_mgmt=1
xen_platform_pci=1
pci_msitranslate=1
viridian=1
hpet=1

############################################################

And the vm produces the following qemu log in /var/log/xen/

############################################################

domid: 4
-videoram option does not work with cirrus vga device model. Videoram set to 4M.
Strip off blktap sub-type prefix to /xen-guests/img/windows7Test.img (drv 'aio')
Using file /xen-guests/img/windows7Test.img in read-write mode
Watching /local/domain/0/device-model/4/logdirty/cmd
Watching /local/domain/0/device-model/4/command
Watching /local/domain/4/cpu
char device redirected to /dev/pts/3
qemu_map_cache_init nr_buckets = 10000 size 4194304
shared page at pfn feffd
buffered io page at pfn feffb
Guest uuid = e09f5a51-7de1-4d92-9cec-826b9d66aeb6
populating video RAM at ff000000
mapping video RAM from ff000000
Register xen platform.
Done register platform.
platform_fixed_ioport: changed ro/rw state of ROM memory area. now is rw state.
xs_read(/local/domain/0/device-model/4/xen_extended_power_mgmt): read error
Log-dirty: no command yet.
I/O request not ready: 0, ptr: 0, port: 0, data: 0, count: 0, size: 0
I/O request not ready: 0, ptr: 0, port: 0, data: 0, count: 0, size: 0
vcpu-set: watch node error.
I/O request not ready: 0, ptr: 0, port: 0, data: 0, count: 0, size: 0
xs_read(/local/domain/4/log-throttling): read error
qemu: ignoring not-understood drive `/local/domain/4/log-throttling'
medium change watch on `/local/domain/4/log-throttling' - unknown device, ignored
I/O request not ready: 0, ptr: 0, port: 0, data: 0, count: 0, size: 0
dm-command: hot insert pass-through pci dev
register_real_device: Assigning real physical device 83:00.0 ...
register_real_device: Enable MSI translation via per device option
register_real_device: Enable power management
pt_iomul_init: Error: pt_iomul_init can't open file /dev/xen/pci_iomul: No such file or directory: 0x83:0x0.0x0
pt_register_regions: IO region registered (size=0x10000000 base_addr=0xe000000c)
pt_register_regions: IO region registered (size=0x00040000 base_addr=0xf0000004)
pt_register_regions: IO region registered (size=0x00000100 base_addr=0x0000f001)
pt_register_regions: Expansion ROM registered (size=0x00020000 base_addr=0xf0040000)
pt_msi_setup: msi mapped with pirq 67
pci_intx: intx=1
register_real_device: Real physical device 83:00.0 registered successfuly!
IRQ type = MSI-INTx
dm-command: hot insert pass-through pci dev
register_real_device: Assigning real physical device 83:00.1 ...
register_real_device: Enable MSI translation via per device option
register_real_device: Enable power management
pt_iomul_init: Error: pt_iomul_init can't open file /dev/xen/pci_iomul: No such file or directory: 0x83:0x0.0x1
pt_register_regions: IO region registered (size=0x00004000 base_addr=0xf0060004)
pt_msi_setup: msi mapped with pirq 66
pci_intx: intx=2
register_real_device: Real physical device 83:00.1 registered successfuly!
IRQ type = MSI-INTx
pt_iomem_map: e_phys=e0000000 maddr=e0000000 type=8 len=268435456 index=0 first_map=1
cirrus vga map change while on lfb mode
pt_iomem_map: e_phys=f3000000 maddr=f0000000 type=0 len=262144 index=2 first_map=1
pt_iomem_map: e_phys=f3060000 maddr=f0060000 type=0 len=16384 index=0 first_map=1
pt_ioport_map: e_phys=c100 pio_base=f000 len=256 index=4 first_map=1
mapping vram to f0000000 - f0400000
platform_fixed_ioport: changed ro/rw state of ROM memory area. now is rw state.
platform_fixed_ioport: changed ro/rw state of ROM memory area. now is ro state.
pt_pci_read_config: [00:04:0] Error: Failed to read register with invalid access size alignment. [Offset:0eh][Length:4]
pt_pci_read_config: [00:04:0] Error: Failed to read register with invalid access size alignment. [Offset:0eh][Length:4]
pt_pci_read_config: [00:04:0] Error: Failed to read register with invalid access size alignment. [Offset:0eh][Length:4]
pt_pci_read_config: [00:04:0] Error: Failed to read register with invalid access size alignment. [Offset:0eh][Length:4]
pt_pci_read_config: [00:04:0] Error: Failed to read register with invalid access size alignment. [Offset:0eh][Length:4]
pt_pci_read_config: [00:04:0] Error: Failed to read register with invalid access size alignment. [Offset:0eh][Length:4]
pt_pci_read_config: [00:04:0] Error: Failed to read register with invalid access size alignment. [Offset:0eh][Length:4]
pt_pci_read_config: [00:05:0] Error: Failed to read register with invalid access size alignment. [Offset:0eh][Length:4]
pt_pci_read_config: [00:05:0] Error: Failed to read register with invalid access size alignment. [Offset:0eh][Length:4]
pt_pci_read_config: [00:05:0] Error: Failed to read register with invalid access size alignment. [Offset:0eh][Length:4]
pt_pci_read_config: [00:05:0] Error: Failed to read register with invalid access size alignment. [Offset:0eh][Length:4]
pt_pci_read_config: [00:05:0] Error: Failed to read register with invalid access size alignment. [Offset:0eh][Length:4]
pt_pci_read_config: [00:05:0] Error: Failed to read register with invalid access size alignment. [Offset:0eh][Length:4]
pt_pci_read_config: [00:05:0] Error: Failed to read register with invalid access size alignment. [Offset:0eh][Length:4]
pt_iomem_map: e_phys=ffffffff maddr=e0000000 type=8 len=268435456 index=0 first_map=0
pt_iomem_map: e_phys=ffffffff maddr=f0000000 type=0 len=262144 index=2 first_map=0
pt_ioport_map: e_phys=ffff pio_base=f000 len=256 index=4 first_map=0
pt_iomem_map: e_phys=e0000000 maddr=e0000000 type=8 len=268435456 index=0 first_map=0
pt_iomem_map: e_phys=f3000000 maddr=f0000000 type=0 len=262144 index=2 first_map=0
pt_ioport_map: e_phys=c100 pio_base=f000 len=256 index=4 first_map=0
pt_iomem_map: e_phys=ffffffff maddr=f0060000 type=0 len=16384 index=0 first_map=0
pt_iomem_map: e_phys=f3060000 maddr=f0060000 type=0 len=16384 index=0 first_map=0
pt_iomem_map: e_phys=ffffffff maddr=e0000000 type=8 len=268435456 index=0 first_map=0
pt_iomem_map: e_phys=ffffffff maddr=f0000000 type=0 len=262144 index=2 first_map=0
pt_ioport_map: e_phys=ffff pio_base=f000 len=256 index=4 first_map=0
pt_iomem_map: e_phys=e0000000 maddr=e0000000 type=8 len=268435456 index=0 first_map=0
pt_iomem_map: e_phys=f3000000 maddr=f0000000 type=0 len=262144 index=2 first_map=0
pt_ioport_map: e_phys=c100 pio_base=f000 len=256 index=4 first_map=0
pt_msgctrl_reg_write: guest enabling MSI, disable MSI-INTx translation
pci_intx: intx=1
pt_msi_disable: Unmap msi with pirq 67
pt_msgctrl_reg_write: setup msi for dev 20
pt_msi_setup: msi mapped with pirq 67
pt_msi_update: Update msi with pirq 67 gvec b0 gflags 130f
pt_iomem_map: e_phys=ffffffff maddr=f0060000 type=0 len=16384 index=0 first_map=0
pt_iomem_map: e_phys=f3060000 maddr=f0060000 type=0 len=16384 index=0 first_map=0

######################################################################

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users





_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users


-- 

--------------------------------------------
Peter Maloney
Brockmann Consult
Max-Planck-Str. 2
21502 Geesthacht
Germany
Tel: +49 4152 889 300
Fax: +49 4152 889 333
E-mail: peter.maloney@xxxxxxxxxxxxxxxxxxxx
Internet: http://www.brockmann-consult.de
--------------------------------------------

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.