Re: [Xen-users] vga passthrogh on mainboards with broken ivrs tables

["Eric Houby" <ehouby@xxxxxxxxx>]

Michael,

Sorry to hear you were not able to get passthrough of your video card
working.  Could you get IOMMU enabled by using no-amd-iommu-perdev-intremap
in the xen command line?  

> (XEN) AMD-Vi: IOMMU 0 Enabled.
> (XEN) AMD-Vi: Enabling global vector map
> (XEN) AMD-Vi: Using global interrupt remap table is not recommended 
> (see XSA-36)!
> (XEN) I/O virtualisation enabled

Thanks for the heads up on irvrs_ioapic.  I will keep an eye out for it in
xen 4.4

Thanks,

Eric

-----Original Message-----
From: xen-users-bounces@xxxxxxxxxxxxx
[mailto:xen-users-bounces@xxxxxxxxxxxxx] On Behalf Of Michaael
Mair-Keimberger
Sent: Tuesday, December 24, 2013 5:42 AM
To: xen-users@xxxxxxxxxxxxx
Subject: Re: [Xen-users] vga passthrogh on mainboards with broken ivrs
tables

Hi Eric,

I still couldn't get it to work. However, since the ivrs_ioapic kernel
parameter didn't do anything (i didn't even got any log messages about
it) i've dug further into this parameter.
Sadly i found out it's not included in xen-4.3.1 and since gentoo doesn't
provide git ebuilds for xen i'll gonna wait until next xen will be released.
xen-4.4 should be release on 21.01.14 - Thats not that far away and i guess
i can wait for it :)

If you want to passthrough your graphic card i suggest you to install a
older version of xen. Personally i have my own xen-4.2.1 ebuild which don't
include XSA-36. That one works perfectly well.
If irvs_ioapic won't help with next xen I'll go back too...

Regards,

Michael

On Sun, Dec 22, 2013 at 06:46:29PM -0700, Eric Houby wrote:
> Michael,
> 
> I have a GA-990FXA-UD3 H/W ver 1.1 that will boot up Xen 4.3.1 (Fedora 
> 20) with IOMMU enabled.
> 
> (XEN) AMD-Vi: IVHD Device Entry: type 0x48 id 0 flags 0xd7
> (XEN) AMD-Vi: IVHD Special: 0000:00:14.0 variety 0x1 handle 0x8
> (XEN) AMD-Vi: IVHD Device Entry: type 0x48 id 0 flags 0xd7
> (XEN) AMD-Vi: IVHD Special: 0000:00:14.0 variety 0x2 handle 0
> (XEN) AMD-Vi: IVHD Device Entry: type 0x48 id 0 flags 0
> (XEN) AMD-Vi: IVHD Special: 0000:00:00.1 variety 0x1 handle 0x8
> (XEN) IVHD Error: Conflicting IO-APIC 0x8 entries
> (XEN) AMD-Vi: IOMMU 0 Enabled.
> (XEN) AMD-Vi: Enabling global vector map
> (XEN) AMD-Vi: Using global interrupt remap table is not recommended 
> (see XSA-36)!
> (XEN) I/O virtualisation enabled
> (XEN)  - Dom0 mode: Relaxed
> (XEN) Interrupt remapping enabled
> (XEN) ENABLING IO-APIC IRQs
> 
> My xen command line is:
> 
> dom0_mem=2048M,max:2048M dom0_max_vcpus=2 dom0_vcpus_pin loglvl=all 
> guest_loglvl=all iommu=debug,verbose,no-amd-iommu-perdev-intremap
>       
> PCI passthrough of USB controllers works without issues, but I have 
> not been able to passthrough a secondary graphics card (HD 5750) 
> without getting a BSoD on my Win7 HVM.  I have had the same 5750 and 
> VM image working on a GA-890FXA, but have not been able to get the 
> combination working on the GA-990FXA.  If you are able to get IOMMU 
> enabled, I will be interested to see if you get the graphics card to
passthrough.
> 
> Good luck.
> 
> -Eric
> 
> 
> 
> 
> -----Original Message-----
> From: xen-users-bounces@xxxxxxxxxxxxx
> [mailto:xen-users-bounces@xxxxxxxxxxxxx] On Behalf Of Michaael 
> Mair-Keimberger
> Sent: Sunday, December 22, 2013 7:08 AM
> To: xen-users@xxxxxxxxxxxxx
> Subject: [Xen-users] vga passthrogh on mainboards with broken ivrs 
> tables
> 
> Hi Lists,
> 
> I'm having troubles getting vga passthrough working again with latest 
> xen-4.3.1.
> 
> My system is a bulldozer amd 8core cpu with a gigabyte GA-990FXA-UD5 
> motherboard.
> The host system uses a amd hd6850 while the guest (windows 7) uses a 
> amd hd 7970.
> 
> So far everything worked great until xsa-36 [1]. I've haven't updated 
> xen since this "fix", but yesterday i'll gave it another try. I've 
> updated to
> xen-4.3.1 and tried to fix those broken ivrs tables with the newly 
> introduced ivrx_ioapic kernel parameter. I'm not sure if this should 
> help me, but at least it sounds promising.
> However i didn't get it to work and i hope someone can give my some 
> hints how to fix that, otherwise i have to downgrade xen again.
> 
> 
> grub2 config:
> 
> menuentry 'Gentoo Linux XEN' {
>         root=hd0,1
>         multiboot /boot/xen.gz dom0_mem=8192M,max:8192M iommu=1
> dom0_max_vcpus=4 dom0_vcpus_pin ivrs_ioapic[8]=00:14.0
>         module /boot/gentoo-3.12.6 root=/dev/md3 radeon.pcie_gen2=1
> radeon.dpm=1 softlevel=xen xen-pciback.permissive
> xen-pciback.hide=(01:00.0)(01:00.1)(00:12.0)(00:12.2)
> }
> 
> lspci:
> 
> 00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI 
> to PCI bridge (external gfx0 port B) (rev 02)
> 00:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD/ATI] RD990 I/O Memory 
> Management Unit (IOMMU)
> 00:02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI 
> to PCI bridge (PCI express gpp port B)
> 00:09.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI 
> to PCI bridge (PCI express gpp port H)
> 00:0a.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI 
> to PCI bridge (external gfx1 port A)
> 00:0d.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI 
> to PCI bridge (external gfx1 port B)
> 00:11.0 SATA controller: Advanced Micro Devices, Inc. [AMD/ATI]
> SB7x0/SB8x0/SB9x0 SATA Controller [AHCI mode] (rev 40)
> 00:12.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI]
> SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
> 00:12.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI]
> SB7x0/SB8x0/SB9x0 USB EHCI Controller
> 00:13.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI]
> SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
> 00:13.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI]
> SB7x0/SB8x0/SB9x0 USB EHCI Controller
> 00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 SMBus 
> Controller (rev 42)
> 00:14.1 IDE interface: Advanced Micro Devices, Inc. [AMD/ATI]
> SB7x0/SB8x0/SB9x0 IDE Controller (rev 40)
> 00:14.2 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 
> Azalia (Intel HDA) (rev 40)
> 00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD/ATI] 
> SB7x0/SB8x0/SB9x0 LPC host controller (rev 40)
> 00:14.4 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 PCI 
> to PCI Bridge (rev 40)
> 00:14.5 USB controller: Advanced Micro Devices, Inc. [AMD/ATI]
> SB7x0/SB8x0/SB9x0 USB OHCI2 Controller
> 00:15.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] 
> SB700/SB800/SB900 PCI to PCI bridge (PCIE port 0)
> 00:15.1 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] 
> SB700/SB800/SB900 PCI to PCI bridge (PCIE port 1)
> 00:15.2 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB900 PCI 
> to PCI bridge (PCIE port 2)
> 00:15.3 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB900 PCI 
> to PCI bridge (PCIE port 3)
> 00:16.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI]
> SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
> 00:16.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI]
> SB7x0/SB8x0/SB9x0 USB EHCI Controller
> 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h 
> Processor Function 0
> 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h 
> Processor Function 1
> 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h 
> Processor Function 2
> 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h 
> Processor Function 3
> 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h 
> Processor Function 4
> 00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h 
> Processor Function 5
> 01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. 
> [AMD/ATI] Tahiti XT [Radeon HD 7970]
> 01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Tahiti XT 
> HDMI Audio [Radeon HD 7970 Series]
> 02:00.0 USB controller: Etron Technology, Inc. EJ168 USB 3.0 Host 
> Controller (rev 01)
> 03:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9172 SATA 
> 6Gb/s Controller (rev 11)
> 04:00.0 VGA compatible controller: Advanced Micro Devices, Inc. 
> [AMD/ATI] Barts PRO [Radeon HD 6850]
> 04:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Barts 
> HDMI Audio [Radeon HD 6800 Series]
> 05:0e.0 FireWire (IEEE 1394): VIA Technologies, Inc. VT6306/7/8 [Fire 
> II(M)] IEEE 1394 OHCI Controller (rev c0)
> 06:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd.
> RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06)
> 07:00.0 USB controller: Etron Technology, Inc. EJ168 USB 3.0 Host 
> Controller (rev 01)
> 08:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9172 SATA 
> 6Gb/s Controller (rev 11)
> 
> lspci -t:
> 
> -[0000:00]-+-00.0
>            +-00.2
>            +-02.0-[01]--+-00.0
>            |            \-00.1
>            +-09.0-[02]----00.0
>            +-0a.0-[03]----00.0
>            +-0d.0-[04]--+-00.0
>            |            \-00.1
>            +-11.0
>            +-12.0
>            +-12.2
>            +-13.0
>            +-13.2
>            +-14.0
>            +-14.1
>            +-14.2
>            +-14.3
>            +-14.4-[05]----0e.0
>            +-14.5
>            +-15.0-[06]----00.0
>            +-15.1-[07]----00.0
>            +-15.2-[08]----00.0
>            +-15.3-[09]--
>            +-16.0
>            +-16.2
>            +-18.0
>            +-18.1
>            +-18.2
>            +-18.3
>            +-18.4
>            \-18.5
> 
> xl dmesg (shorten):
> 
> (XEN) Xen version 4.3.1 (@xonet.at) (x86_64-pc-linux-gnu-gcc (Gentoo 
> 4.8.2 p1.0, pie-0.5.8) 4.8.2) debug=n Sat Dec 21 11:55:39 CET 2013
> (XEN) Latest ChangeSet: 
> (XEN) Bootloader: GRUB 2.02~beta1
> (XEN) Command line: dom0_mem=8192M,max:8192M iommu=1 dom0_max_vcpus=4 
> dom0_vcpus_pin ivrs_ioapic[8]=00:14.0 ...
> (XEN) IOAPIC[0]: apic_id 8, version 33, address 0xfec00000, GSI 0-23
> (XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
> (XEN) Using scheduler: SMP Credit Scheduler (credit)
> (XEN) Detected 4026.984 MHz processor.
> (XEN) Initing memory sharing.
> (XEN) xstate_init: using cntxt_size: 0x3c0 and states: 
> 0x4000000000000007
> (XEN) IVHD Error: Conflicting IO-APIC 0x8 entries
> (XEN) AMD-Vi: Error initialization
> (XEN) I/O virtualisation disabled
> (XEN) ENABLING IO-APIC IRQs
> (XEN)  -> Using new ACK method
> (XEN) Platform timer is 14.318MHz HPET
> (XEN) Allocated console ring of 16 KiB.
> (XEN) HVM: ASIDs enabled.
> (XEN) SVM: Supported advanced features:
> (XEN)  - Nested Page Tables (NPT)
> (XEN)  - Last Branch Record (LBR) Virtualisation
> (XEN)  - Next-RIP Saved on #VMEXIT
> (XEN)  - VMCB Clean Bits
> (XEN)  - DecodeAssists
> (XEN)  - Pause-Intercept Filter
> (XEN)  - TSC Rate MSR
> (XEN) HVM: SVM enabled
> (XEN) HVM: Hardware Assisted Paging (HAP) detected
> (XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
> (XEN) spurious 8259A interrupt: IRQ7.
> 
> 
> I've tried a few other configuration's (like adding the ivrs_ioapic 
> parameter to the module line in grub or also adding
> vfio_iommu_type1.allow_unsafe_interrupts=1
> to the cmdline), but so far i couldn't get it to work.
> 
> Since i'm not completely sure what i actually have to insert at the 
> kernel parameter i'm glad if someone could give me some insight. 
> However, since the output of xl dmesg is always the same, even though 
> with or without the parameter, i might missed some additional parameters.
> 
> If you need any more information, I'll be glad to provide them.
> 
> Thanks in advance.
> 
> Michael
> 
> 
> 
> 
> 
> 
> 
> 
> [1] 
> http://lists.xen.org/archives/html/xen-announce/2013-02/msg00002.html
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@xxxxxxxxxxxxx
> http://lists.xen.org/xen-users
> 

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users