Re: [Xen-users] Help with VGA passthrough

[Gordan Bobic <gordan@xxxxxxxxxx>]

I'm pretty sure this is the wrong list for Xenserver/Xencenter.

Having said that, look at the configuration file generated. If it includes 
gfx_passthru=1, remove it. I'm pretty sure this doesn't work. What most people 
is is secondary pass through by passing the card through as a normal PCI 
device, with initial boot-up output on VNC.

In terms of device capture, reboot the machine and look at lspci -vvv. If 
anything other than pciback is the driver attached to your GPU, it isn't going 
to work. If the radeon driver touched the card it has been tainted and until 
the bus reset method is implemented it won't work. You will need to remove the 
radeon driver from your initrd, blacklist it in /etc/modprobe.d/, create a 
configuration file for xen-pciback in there to make it capture the PCI devices 
you will be passing through, and create a script in /etc/sysconfig/modules/ to 
modprobe xen-pciback and AFTER that modprobe radeon if your primary GPU also 
uses that driver.

Note that I have seen very bizzare interaction between different generation ATI 
cards when it comes to virtualization. A 7970+6450 seemed to work fine, but 
IIRC if I used a 4850 as primary for domU things would inexplicably break. I 
think there was something weird happening along the lines of ATI BIOS of the 
primary card trying to initialise (some of the) other cards as well, and the 
4850 BIOS makes incorrect assumptions about later cards.

While I had reasonable degree of success passing a 6450 through and reboots 
worked most of the time it still never was anywhere nearly as trouble free as 
working with (modified) Nvidia cards. Then again, I could say the same thing 
about running on bare metal...


Brian Menges <bmenges@xxxxxxxxxx> wrote:

>I'm trying to get vga passthrough to operate normally and have been rather 
>unsuccessful despite attempting to replicate hardware stated to work much 
>reading about the mail list and wiki.
>
>Setup:
>Xenserver 6.2 SP1 (all patches)
>ATI Radeon HD 6450
>ThinkCentre M series with Core i5
>
>I've read:
>http://support.citrix.com/article/CTX135811http://support.citrix.com/article/CTX135811
>http://wiki.xen.org/wiki/Xen_VGA_Passthrough
>
>The GPU assignment to the HVM works fine through both api and gui (xencenter), 
>however the domU can't initialize it. I've tried pciback.hide and 
>xen-pciback.hide for the pci buses the ati card shows up as, however to no 
>avail and results in it unable to use the gpu.
>
>Dmesg about the pci bus (01:00.0 and 01:00.1):
>[    4.225073] Kernel command line: root=LABEL=root-lsodryqz ro xencons=hvc 
>console=hvc0 console=tty0 quiet vga=785 splash xen-pciback.permissive 
>xen-pciback.hide=(01:00.0)(01:00.1) pci=resource_alignment=01:00.0;01:00.1
>[    4.790356] pci 0000:01:00.0: reg 10 64bit mmio pref: 
>[0xd0000000-0xdfffffff]
>[    4.790369] pci 0000:01:00.0: reg 18 64bit mmio: [0xfe420000-0xfe43ffff]
>[    4.790376] pci 0000:01:00.0: reg 20 io port: [0xe000-0xe0ff]
>[    4.790389] pci 0000:01:00.0: reg 30 32bit mmio pref: 
>[0xfe400000-0xfe41ffff]
>[    4.790394] pci 0000:01:00.0: Disabling memory decoding and releasing 
>memory resources.
>[    4.790425] pci 0000:01:00.0: supports D1 D2
>[    4.790479] pci 0000:01:00.1: reg 10 64bit mmio: [0xfe440000-0xfe443fff]
>[    4.790513] pci 0000:01:00.1: Disabling memory decoding and releasing 
>memory resources.
>[    4.790543] pci 0000:01:00.1: supports D1 D2
>[    4.794119] vgaarb: device added: 
>PCI:0000:01:00.0,decodes=io+mem,owns=none,locks=none
>
>When I assign it to a linux domU (yes, HVM):
>00:05.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] 
>Caicos [Radeon HD 6450/7450/8450 / R5 230 OEM]
>00:06.0 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Caicos HDMI Audio 
>[Radeon HD 6400 Series]
>
>[    5.246901] [drm] initializing kernel modesetting (CAICOS 0x1002:0x6779 
>0x1682:0x3207).
>[    5.246970] [drm] register mmio base: 0xF3000000
>[    5.246972] [drm] register mmio size: 131072
>[    5.246996] radeon 0000:00:05.0: Expecting atombios for evergreen GPU
>[    5.247053] radeon 0000:00:05.0: Fatal error during GPU init
>[    5.247100] [drm] radeon: finishing device.
>[    5.247102] [TTM] Memory type 2 has not been initialized
>[    5.256465] radeon: probe of 0000:00:05.0 failed with error -22
>[    5.386743] xen: --> pirq=17 -> irq=41 (gsi=41)
>
>Here's dom0's extboot.conf for xe's boot:
>label xe
>  # XenServer
>  kernel mboot.c32
>  append /boot/xen.gz mem=1024G dom0_max_vcpus=4 dom0_mem=2048M,max:2048M 
> watchdog_timeout=300 lowmem_emergency_pool=1M crashkernel=64M@32M 
> cpuid_mask_xsave_eax=0 console=vga vga=mode-0x0311 --- /boot/vmlinuz-2.6-xen 
> root=LABEL=root-lsodryqz ro xencons=hvc console=hvc0 console=tty0 quiet 
> vga=785 splash xen-pciback.permissive xen-pciback.hide=(01:00.0)(01:00.1) 
> pci=resource_alignment=01:00.0;01:00.1 --- /boot/initrd-2.6-xen.img
>
>So I've hidden them, however they're still showing up on dom0 and failing init 
>on domU.
>
>Can anyone provide some help?
>
>- Brian Menges
>Principal Engineer, DevOps
>GoGrid | ServePath | ColoServe | UpStream Networks
>
>________________________________
>
>The information contained in this message, and any attachments, may contain 
>confidential and legally privileged material. It is solely for the use of the 
>person or entity to which it is addressed. Any review, retransmission, 
>dissemination, or action taken in reliance upon this information by persons or 
>entities other than the intended recipient is prohibited. If you receive this 
>in error, please contact the sender and delete the material from any computer.
>
>_______________________________________________
>Xen-users mailing list
>Xen-users@xxxxxxxxxxxxx
>http://lists.xen.org/xen-users
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users