Re: [Xen-users] Xen IOMMU disabled due to IVRS table... Blah blah blah

[Jan Hejl <jh@xxxxxxxxxx>]

Hi guys,

i'm also experiencing this problem with M5A97 EVO R2.0. It has been "solved" in Xen 4.2.2 (discused here http://lists.xen.org/archives/html/xen-announce/2013-02/msg00006.html). But i agree on that that it should be repaired on ASUS side.

And I'm also facing with prolem about Adaptec 3805 Raid card with enabled IOMMU on this motherbord. Without IOMMU storage adapter works perfectly, but with enabled IOMMU in BIOS i see the device under linux as block device but it's unusable - no data can read or written. Does anyone know what to do here, please? Could it be related with bad IOMMU implementation?

Thanks in advance
Jan

Dne 28.5.2013 12:11, Gizmo Chicken napsal(a):

David and feral,

I have an Asus M5A99FX Pro 2.0 motherboard and, at least as far as I can tell, IOMMU works well with that motherboard and Xen 4.1.  (I got VGA passthrough working with a Windows guest.)   Although I did some preliminary testing with Xen 4.2 on that motherboard, because of some other issues, I never got around to creating a VM that required IOMMU support, and so can't say for certain whether IOMMU is working with that motherboard and Xen 4.2.

Apart from creating a VM that requires IOMMU support (which would take more time than I'd prefer to spend right now) to see if it works, what would good way to determine whether my  Asus M5A99FX Pro 2.0 motherboard suffers, or if free from, the IVRS table issue that you describe?

Thanks much for any help with this!

Best regards,

GizmoChicken

On Tue, May 28, 2013 at 3:31 AM, David Sutton <kantras@xxxxxxxxx> wrote:

Feral,

  I have the R1.0 version of that motherboard. The problem is with the BIOS; its returning bad information in the IVRS table for the IO-APICs, which the new parser is catching and causing AMD-Vi initialization to fail. I've already put in a support ticket via the online form, included all the information such as links to the document which shows what the valid details should be. The ticket was basically closed with a note saying to look out for BIOS updates. I'm currently using 4.2.1 (which was before the table checking was added) and am able to use AMD-Vi. If you want to fix this, you're either going to have to get ASUS to issue a fixed BIOS (which doesn't feel likely at this point) or you'd have to either disable interrupt remapping or edit the source code to disable that check (and deal with any potential issue that might cause later)

Regards,

  David

On Mon, May 27, 2013 at 11:42 PM, feral <blistovmhz@xxxxxxxxx> wrote:

This issue has been discussed many times before, but I haven't found
an answer in my specific problem.

Motherboard: Asus Sabertooth 990fx R2.0
Linux 3.2.0-35 through 3.8.x (tested 5 kernels to be ridiculously thorough)

Affected: Xen-hypervisor >4.1.2
Not Affected: Xen-hypervisor <4.1.2


Xen 4.1.2 and earlier works fine regardless of the BIOS version of the
Mobo.  After upgrading Xen hypervisor beyond 4.1.2, we get the
following:
(XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, GSI 0-23
(XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, GSI 24-55
(XEN) Enabling APIC mode:  Flat.  Using 2 I/O APICs
(XEN) Table is not found!
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 3110.540 MHz processor.
(XEN) Initing memory sharing.
(XEN) xstate_init: using cntxt_size: 0x3c0 and states: 0x4000000000000007
(XEN) PCI: Not using MCFG for segment 0000 bus 00-ff
(XEN) IVHD Error: Invalid IO-APIC 0xff
(XEN) AMD-Vi: Error initialization
(XEN) I/O virtualisation disabled


From what I understand, Xen used to just care less about the state of
your ACPI/IVRS but that this was potentially a security concern and
checks were implemented to disable IOMMU whenever it would be unsafe
(potentially) to have it enabled.
Now, I understand the logic here, but I disagree with the
implementation.  There are a fairly large number of people who are
affected and we know Asus isn't going to fix the IVRS table.  I
recognize the importance of implementing these checks and hell, even
automatically disabling IOMMU without notice (though on a consumer
board, this is a bit paranoid).
I understand there are kernel command line options you can pass to
forcibly bypass the checks and leave IOMMU enabled in most cases (ie:
iommu=no-amd-iommu-perdev-intremap) but thus far none of them seem to
be enough to force IOMMU back on.
I read somewhere in one of the many discussions on the subject, that
there are cases where you simply aren't allowed to disable the checks
(I believe this was when the IVRS table indicies don't line up or
something, which seems to be my issue).

As I said, I know other people have precisely this problem with a
whole line of mid range Asus boards, but haven't seen a lot of
discussion on the list.  Anyone reading this run into the same
problem?  I bought this board specifically because of it's IOMMU
implementation, which worked fine... at the time...  Ultimately I know
for a  fact that the boards ACPI/IVRS can't be TOO wrong as it's been
working for 6 months without an issue.  I'm just sorta in a situation
where it's time to upgrade my base OS and hacking Xen 4.1.2 in seems
like a silly amount of work when all I really need is an option to
disable the checks.


xm dmesg from a working vs. borked system
Working:
(XEN) Xen version 4.1.2 (Ubuntu 4.1.2-2ubuntu2)
(stefan.bader@xxxxxxxxxxxxx) (gcc version 4.6.3 (Ubuntu/Linaro
4.6.3-1ubuntu2) ) Thu Mar  8 14:31:11 UTC 2012
(XEN) Bootloader: GRUB 1.99-21ubuntu3.9
(XEN) Command line: pci_msitranslate=0 xen-pciback=passthrough
xen-pciback.hide=(06:00.0)(06.00.1)
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN)  Found 6 MBR signatures
(XEN)  Found 6 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 000000000009e800 (usable)
(XEN)  000000000009e800 - 00000000000a0000 (reserved)
(XEN)  00000000000e0000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 00000000bc348000 (usable)
(XEN)  00000000bc348000 - 00000000bc78c000 (reserved)
(XEN)  00000000bc78c000 - 00000000bc797000 (ACPI data)
(XEN)  00000000bc797000 - 00000000bd4f8000 (ACPI NVS)
(XEN)  00000000bd4f8000 - 00000000bd8d5000 (reserved)
(XEN)  00000000bd8d5000 - 00000000bd8d6000 (usable)
(XEN)  00000000bd8d6000 - 00000000bdad9000 (ACPI NVS)
(XEN)  00000000bdad9000 - 00000000bdf00000 (usable)
(XEN)  00000000f8000000 - 00000000fc000000 (reserved)
(XEN)  00000000fec00000 - 00000000fec01000 (reserved)
(XEN)  00000000fec10000 - 00000000fec11000 (reserved)
(XEN)  00000000fec20000 - 00000000fec21000 (reserved)
(XEN)  00000000fed00000 - 00000000fed01000 (reserved)
(XEN)  00000000fed61000 - 00000000fed71000 (reserved)
(XEN)  00000000fed80000 - 00000000fed90000 (reserved)
(XEN)  00000000fef00000 - 0000000100000000 (reserved)
(XEN)  0000000100001000 - 000000043f000000 (usable)
(XEN) ACPI: RSDP 000F0490, 0024 (r2 ALASKA)
(XEN) ACPI: XSDT BC78E078, 0064 (r1 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: FACP BC795BA8, 00F4 (r4 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI Warning (tbfadt-0444): Optional field "Pm2ControlBlock" has
zero address or length: 0000000000000000/1 [20070126]
(XEN) ACPI: DSDT BC78E170, 7A38 (r2 ALASKA    A M I        0 INTL 20051117)
(XEN) ACPI: FACS BD4F2F80, 0040
(XEN) ACPI: APIC BC795CA0, 009E (r3 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: FPDT BC795D40, 0044 (r1 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: MCFG BC795D88, 003C (r1 ALASKA    A M I  1072009 MSFT    10013)
(XEN) ACPI: HPET BC795DC8, 0038 (r1 ALASKA    A M I  1072009 AMI         5)
(XEN) ACPI: BGRT BC796260, 0038 (r0 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: IVRS BC795E58, 0100 (r1  AMD     RD890S   202031 AMD         0)
(XEN) ACPI: SSDT BC795F58, 0304 (r1 AMD    POWERNOW        1 AMD         1)
(XEN) System RAM: 16311MB (16702516kB)
(XEN) Domain heap initialised
(XEN) ACPI: 32/64X FACS address mismatch in FADT -
bd4f2f80/0000000000000000, using 32
(XEN) Processor #16 5:1 APIC version 16
(XEN) Processor #17 5:1 APIC version 16
(XEN) Processor #18 5:1 APIC version 16
(XEN) Processor #19 5:1 APIC version 16
(XEN) Processor #20 5:1 APIC version 16
(XEN) Processor #21 5:1 APIC version 16
(XEN) Processor #22 5:1 APIC version 16
(XEN) Processor #23 5:1 APIC version 16
(XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, GSI 0-23
(XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, GSI 24-55
(XEN) Enabling APIC mode:  Flat.  Using 2 I/O APICs
(XEN) Table is not found!
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 3110.553 MHz processor.
(XEN) Initing memory sharing.
(XEN) AMD-Vi: IOMMU 0 Enabled.
(XEN) I/O virtualisation enabled
(XEN)  - Dom0 mode: Relaxed
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using new ACK method
(XEN) Platform timer is 14.318MHz HPET
(XEN) Allocated console ring of 16 KiB.
(XEN) HVM: ASIDs enabled.
(XEN) SVM: Supported advanced features:
(XEN)  - Nested Page Tables (NPT)
(XEN)  - Last Branch Record (LBR) Virtualisation
(XEN)  - Next-RIP Saved on #VMEXIT
(XEN)  - VMCB Clean Bits
(XEN)  - Pause-Intercept Filter
(XEN) HVM: SVM enabled
(XEN) HVM: Hardware Assisted Paging detected.
(XEN) Brought up 8 CPUs
(XEN) *** LOADING DOMAIN 0 ***
(XEN)  Xen  kernel: 64-bit, lsb, compat32
(XEN)  Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x205f000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   0000000420000000->0000000428000000 (4044532
pages to be allocated)
(XEN)  Init. ramdisk: 000000043c7dd000->000000043efffa00
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: ffffffff81000000->ffffffff8205f000
(XEN)  Init. ramdisk: ffffffff8205f000->ffffffff84881a00
(XEN)  Phys-Mach map: ffffffff84882000->ffffffff867b18b8
(XEN)  Start info:    ffffffff867b2000->ffffffff867b24b4
(XEN)  Page tables:   ffffffff867b3000->ffffffff867ec000
(XEN)  Boot stack:    ffffffff867ec000->ffffffff867ed000
(XEN)  TOTAL:         ffffffff80000000->ffffffff86c00000
(XEN)  ENTRY ADDRESS: ffffffff81cfb200
(XEN) Dom0 has maximum 8 VCPUs
(XEN) Scrubbing Free RAM: .done.
(XEN) Xen trace buffers: disabled
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type \047CTRL-a\047 three times to
switch input to Xen)
(XEN) Freed 220kB init memory.
(XEN) traps.c:2432:d0 Domain attempted WRMSR 00000000c0010201 from
0x0000000000000000 to 0x000000000000abcd.
(XEN) physdev.c:155: dom0: wrong map_pirq type 3


BORKED:
(XEN) Xen version 4.2.1 (Ubuntu 4.2.1-0ubuntu3.1)
(stefan.bader@xxxxxxxxxxxxx) (gcc (Ubuntu/Linaro 4.7.3-1ubuntu1)
4.7.3) Mon Apr 29 19:35:31 UTC 2013
(XEN) Bootloader: GRUB 2.00-13ubuntu3
(XEN) Command line: iommu=no-amd-iommu-perdev-intremap x2apic=off
xen-pciback.hide=(06:00.0)(06.00.1)
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN)  Found 7 MBR signatures
(XEN)  Found 6 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 000000000009e800 (usable)
(XEN)  000000000009e800 - 00000000000a0000 (reserved)
(XEN)  00000000000e0000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 00000000ba7ac000 (usable)
(XEN)  00000000ba7ac000 - 00000000babe0000 (reserved)
(XEN)  00000000babe0000 - 00000000babf0000 (ACPI data)
(XEN)  00000000babf0000 - 00000000bb958000 (ACPI NVS)
(XEN)  00000000bb958000 - 00000000bca35000 (reserved)
(XEN)  00000000bca35000 - 00000000bca36000 (usable)
(XEN)  00000000bca36000 - 00000000bcc3c000 (ACPI NVS)
(XEN)  00000000bcc3c000 - 00000000bd083000 (usable)
(XEN)  00000000bd083000 - 00000000bd7f4000 (reserved)
(XEN)  00000000bd7f4000 - 00000000bd800000 (usable)
(XEN)  00000000f8000000 - 00000000fc000000 (reserved)
(XEN)  00000000fec00000 - 00000000fec01000 (reserved)
(XEN)  00000000fec10000 - 00000000fec11000 (reserved)
(XEN)  00000000fec20000 - 00000000fec21000 (reserved)
(XEN)  00000000fed00000 - 00000000fed01000 (reserved)
(XEN)  00000000fed61000 - 00000000fed71000 (reserved)
(XEN)  00000000fed80000 - 00000000fed90000 (reserved)
(XEN)  00000000fef00000 - 0000000100000000 (reserved)
(XEN)  0000000100001000 - 000000043f000000 (usable)
(XEN) ACPI: RSDP 000F0490, 0024 (r2 ALASKA)
(XEN) ACPI: XSDT BABE7078, 0064 (r1 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: FACP BABEE118, 010C (r5 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI Warning (tbfadt-0232): FADT (revision 5) is longer than
ACPI 2.0 version, truncating length 0x10C to 0xF4 [20070126]
(XEN) ACPI Warning (tbfadt-0444): Optional field "Pm2ControlBlock" has
zero address or length: 0000000000000000/1 [20070126]
(XEN) ACPI: DSDT BABE7170, 6FA8 (r2 ALASKA    A M I        0 INTL 20051117)
(XEN) ACPI: FACS BB952F80, 0040
(XEN) ACPI: APIC BABEE228, 009E (r3 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: FPDT BABEE2C8, 0044 (r1 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: MCFG BABEE310, 003C (r1 ALASKA    A M I  1072009 MSFT    10013)
(XEN) ACPI: HPET BABEE350, 0038 (r1 ALASKA    A M I  1072009 AMI         5)
(XEN) ACPI: BGRT BABEFBF8, 0038 (r0 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: IVRS BABEE3E0, 0100 (r1  AMD     RD890S   202031 AMD         0)
(XEN) ACPI: SSDT BABEE4E0, 1714 (r1 AMD    POWERNOW        1 AMD         1)
(XEN) System RAM: 16283MB (16674420kB)
(XEN) Domain heap initialised
(XEN) ACPI: 32/64X FACS address mismatch in FADT -
bb952f80/0000000000000000, using 32
(XEN) Processor #16 5:1 APIC version 16
(XEN) Processor #17 5:1 APIC version 16
(XEN) Processor #18 5:1 APIC version 16
(XEN) Processor #19 5:1 APIC version 16
(XEN) Processor #20 5:1 APIC version 16
(XEN) Processor #21 5:1 APIC version 16
(XEN) Processor #22 5:1 APIC version 16
(XEN) Processor #23 5:1 APIC version 16
(XEN) IOAPIC[0]: apic_id 9, version 33, address 0xfec00000, GSI 0-23
(XEN) IOAPIC[1]: apic_id 10, version 33, address 0xfec20000, GSI 24-55
(XEN) Enabling APIC mode:  Flat.  Using 2 I/O APICs
(XEN) Table is not found!
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 3110.540 MHz processor.
(XEN) Initing memory sharing.
(XEN) xstate_init: using cntxt_size: 0x3c0 and states: 0x4000000000000007
(XEN) PCI: Not using MCFG for segment 0000 bus 00-ff
(XEN) IVHD Error: Invalid IO-APIC 0xff
(XEN) AMD-Vi: Error initialization
(XEN) I/O virtualisation disabled
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using new ACK method
(XEN) Platform timer is 14.318MHz HPET
(XEN) Allocated console ring of 16 KiB.
(XEN) HVM: ASIDs enabled.
(XEN) SVM: Supported advanced features:
(XEN)  - Nested Page Tables (NPT)
(XEN)  - Last Branch Record (LBR) Virtualisation
(XEN)  - Next-RIP Saved on #VMEXIT
(XEN)  - VMCB Clean Bits
(XEN)  - DecodeAssists
(XEN)  - Pause-Intercept Filter
(XEN)  - TSC Rate MSR
(XEN) HVM: SVM enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected
(XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
(XEN) Brought up 8 CPUs
(XEN) mtrr: your CPUs had inconsistent variable MTRR settings
(XEN) *** LOADING DOMAIN 0 ***
(XEN)  Xen  kernel: 64-bit, lsb, compat32
(XEN)  Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x2346000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   0000000420000000->0000000428000000 (4035583
pages to be allocated)
(XEN)  Init. ramdisk: 0000000439dc6000->000000043efff800
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: ffffffff81000000->ffffffff82346000
(XEN)  Init. ramdisk: ffffffff82346000->ffffffff8757f800
(XEN)  Phys-Mach map: ffffffff87580000->ffffffff894b31c8
(XEN)  Start info:    ffffffff894b4000->ffffffff894b44b4
(XEN)  Page tables:   ffffffff894b5000->ffffffff89504000
(XEN)  Boot stack:    ffffffff89504000->ffffffff89505000
(XEN)  TOTAL:         ffffffff80000000->ffffffff89800000
(XEN)  ENTRY ADDRESS: ffffffff81d06210
(XEN) Dom0 has maximum 8 VCPUs
(XEN) Scrubbing Free RAM: .done.
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type \047CTRL-a\047 three times to
switch input to Xen)
(XEN) Freed 244kB init memory.

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users

Attachment: smime.p7s
Description: Elektronicky podpis S/MIME

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users