Re: [Xen-devel] VT-d issue on Xen 3.3.0 [WAS: Xen-3.2.1 VT-d Support (NOT SURE WHETHER IT'S ABUGOR...)]

["Jimmy Jin" <jimmyjin.maillist@xxxxxxxxx>]

Hello, all,

The post on HP's support forum was from me. So I'm also quite
interested to know when there is new BIOS available which fixes this
problem.

Thanks,
Jimmy Jin

2008/9/11 Cui, Dexuan <dexuan.cui@xxxxxxxxx>:
> Hi Ed,
> Thanks for the report!
> Looks this is a BIOS issue. I think a new BIOS should be released soon. Will
> keep you informed.
>
> Thanks!
>
> -- Dexuan
>
> ________________________________
> From: Nadolski, Ed [mailto:Ed.Nadolski@xxxxxxx]
> Sent: 2008年9月11日 6:49
> To: Nadolski, Ed; Han, Weidong; Kumar, Venkat; Cui, Dexuan; Bryan York
> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
> Subject: RE: [Xen-devel] VT-d issue on Xen 3.3.0 [WAS: Xen-3.2.1 VT-d
> Support (NOT SURE WHETHER IT'S ABUGOR...)]
>
> I have more info:  when I flash to an older BIOS, version 0559 (this
> required the recovery procedure) the VT-d seems to come up OK.  The xm dmesg
> is below.
>
>
>
> I'm surprised the old BIOS works better.  So be careful about upgrading your
> BIOS.  Any thoughts on what the bug might be?  Is there anywhere I can
> report this to Intel?
>
>
>
> Thanks,
>
> Ed
>
>
>
>
>
> (XEN) Xen EXPERIMENTAL TEST BUILD
>
> (XEN) Xen version 3.3.0 (root@) (gcc version 4.1.2 20071124 (Red Hat
> 4.1.2-42)) Wed Sep 10 14:43:01 MDT 2008
>
> (XEN) Latest ChangeSet: unavailable
>
> (XEN) Xen EXPERIMENTAL TEST BUILD!
>
> (XEN) Command line: vtd=1 iommu=1 acpi=force apic=on
>
> (XEN) Video information:
>
> (XEN)  VGA is text mode 80x25, font 8x16
>
> …
>
> (XEN) Processor #0 6:15 APIC version 20
>
> (XEN) Processor #1 6:15 APIC version 20
>
> (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
>
> (XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
>
> (XEN) [VT-D]dmar.c:446: Host address width 36
>
> (XEN) [VT-D]dmar.c:450: ED_DEBUG: dmar:0xffff828bfff7b000
>
> (XEN) [VT-D]dmar.c:451: ED_DEBUG: entry_header:0xffff828bfff7b030
>
> (XEN) [VT-D]dmar.c:452: ED_DEBUG: table->length:296
>
> (XEN) [VT-D]dmar.c:458: ED_DEBUG: entry_header:0xffff828bfff7b030 type:0x0
> length:24
>
> (XEN) [VT-D]dmar.c:463: found ACPI_DMAR_DRHD
>
> (XEN) [VT-D]dmar.c:323: dmaru->address = feb00000
>
> (XEN) [VT-D]dmar.c:272: found endpoint: bdf = 0:1b.0
>
> (XEN) [VT-D]dmar.c:458: ED_DEBUG: entry_header:0xffff828bfff7b048 type:0x0
> length:32
>
> (XEN) [VT-D]dmar.c:463: found ACPI_DMAR_DRHD
>
> (XEN) [VT-D]dmar.c:323: dmaru->address = feb01000
>
> (XEN) [VT-D]dmar.c:272: found endpoint: bdf = 0:2.0
>
> (XEN) [VT-D]dmar.c:272: found endpoint: bdf = 0:2.1
>
> (XEN) [VT-D]dmar.c:458: ED_DEBUG: entry_header:0xffff828bfff7b068 type:0x0
> length:48
>
> (XEN) [VT-D]dmar.c:463: found ACPI_DMAR_DRHD
>
> (XEN) [VT-D]dmar.c:323: dmaru->address = feb02000
>
> (XEN) [VT-D]dmar.c:272: found endpoint: bdf = 0:3.0
>
> (XEN) [VT-D]dmar.c:272: found endpoint: bdf = 0:3.1
>
> (XEN) [VT-D]dmar.c:272: found endpoint: bdf = 0:3.2
>
> (XEN) [VT-D]dmar.c:272: found endpoint: bdf = 0:3.3
>
> (XEN) [VT-D]dmar.c:458: ED_DEBUG: entry_header:0xffff828bfff7b098 type:0x0
> length:16
>
> (XEN) [VT-D]dmar.c:463: found ACPI_DMAR_DRHD
>
> (XEN) [VT-D]dmar.c:323: dmaru->address = feb03000
>
> (XEN) [VT-D]dmar.c:332: found INCLUDE_ALL
>
> (XEN) [VT-D]dmar.c:458: ED_DEBUG: entry_header:0xffff828bfff7b0a8 type:0x1
> length:88
>
> (XEN) [VT-D]dmar.c:467: found ACPI_DMAR_RMRR
>
> (XEN) [VT-D]dmar.c:359: ED_DEBUG: RMRR base:0x00000000000e0000  RMRR
> end:0x00000000000effff
>
> (XEN) [VT-D]dmar.c:272: found endpoint: bdf = 0:1d.0
>
> (XEN) [VT-D]dmar.c:272: found endpoint: bdf = 0:1d.1
>
> (XEN) [VT-D]dmar.c:272: found endpoint: bdf = 0:1d.2
>
> (XEN) [VT-D]dmar.c:272: found endpoint: bdf = 0:1d.7
>
> (XEN) [VT-D]dmar.c:272: found endpoint: bdf = 0:1a.0
>
> (XEN) [VT-D]dmar.c:272: found endpoint: bdf = 0:1a.1
>
> (XEN) [VT-D]dmar.c:272: found endpoint: bdf = 0:1a.2
>
> (XEN) [VT-D]dmar.c:272: found endpoint: bdf = 0:1a.7
>
> (XEN) [VT-D]dmar.c:458: ED_DEBUG: entry_header:0xffff828bfff7b100 type:0x1
> length:40
>
> (XEN) [VT-D]dmar.c:467: found ACPI_DMAR_RMRR
>
> (XEN) [VT-D]dmar.c:359: ED_DEBUG: RMRR base:0x00000000cf600000  RMRR
> end:0x00000000cfffffff
>
> (XEN) [VT-D]dmar.c:272: found endpoint: bdf = 0:2.0
>
> (XEN) [VT-D]dmar.c:272: found endpoint: bdf = 0:2.1
>
> (XEN) Intel VT-d has been enabled
>
>
>
>
>
> ________________________________
>
> From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
> [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Nadolski, Ed
> Sent: Wednesday, September 10, 2008 1:56 PM
> To: Han, Weidong; Kumar, Venkat; Cui, Dexuan; Bryan York
> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
> Subject: [Xen-devel] VT-d issue on Xen 3.3.0 [WAS: Xen-3.2.1 VT-d Support
> (NOT SURE WHETHER IT'S ABUGOR...)]
>
>
>
> I am seeing the "RMRR is incorrect" issue with Xen 3.3.0 and CentOS 5.2 on a
> DQ35 with the latest BIOS (JOQ3510J.86A.0942.2008.0807.1958).   It happens
> with either IGD or PEG.  CPU is E6750, main memory is 4Gb. Debug trace is
> below.
>
>
>
> I'm not sure what the RMRR entries should look like.  What BIOS revision
> worked for you?  Should I change any other BIOS settings?   Current video
> settings are:
>
>
>
> DVMT Mode = DVMT
>
> IGD DVMT Memory = 128Mb
>
> IGD Aperture Size = 256Mb
>
>
>
> Are there any other things that would be good to try?
>
>
>
> Thanks,
>
> Ed Nadolski
>
> ed.nadolski@xxxxxxx
>
>
>
>
>
> (XEN) Xen version 3.3.0 (root@) (gcc version 4.1.2 20071124 (Red Hat
> 4.1.2-42)) Tue Sep  9 17:36:19 MDT 2008
>
> (XEN) Latest ChangeSet: unavailable
>
> (XEN) Xen EXPERIMENTAL TEST BUILD!
>
> (XEN) Command line: vtd=1 iommu=1 acpi=force apic=on
>
> (XEN) Video information:
>
> (XEN)  VGA is text mode 80x25, font 8x16
>
> (XEN)  VBE/DDC methods: V2; EDID transfer time: 1 seconds
>
> ……
>
> (XEN) Processor #0 6:15 APIC version 20
>
> (XEN) Processor #1 6:15 APIC version 20
>
> (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
>
> (XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
>
> (XEN) [VT-D]dmar.c:447: Host address width 36
>
> (XEN) [VT-D]dmar.c:456: found ACPI_DMAR_DRHD
>
> (XEN) [VT-D]dmar.c:323: dmaru->address = feb00000
>
> (XEN) [VT-D]dmar.c:272: found endpoint: bdf = 0:1b.0
>
> (XEN) [VT-D]dmar.c:456: found ACPI_DMAR_DRHD
>
> (XEN) [VT-D]dmar.c:323: dmaru->address = feb01000
>
> (XEN) [VT-D]dmar.c:272: found endpoint: bdf = 0:2.0
>
> (XEN) [VT-D]dmar.c:272: found endpoint: bdf = 0:2.1
>
> (XEN) [VT-D]dmar.c:456: found ACPI_DMAR_DRHD
>
> (XEN) [VT-D]dmar.c:323: dmaru->address = feb03000
>
> (XEN) [VT-D]dmar.c:332: found INCLUDE_ALL
>
> (XEN) [VT-D]dmar.c:460: found ACPI_DMAR_RMRR
>
> (XEN) [VT-D]dmar.c:359: ED_DEBUG: RMRR base:0x00000000000e0000  RMRR
> end:0x00000000000effff
>
> (XEN) [VT-D]dmar.c:272: found endpoint: bdf = 0:1d.0
>
> (XEN) [VT-D]dmar.c:272: found endpoint: bdf = 0:1d.1
>
> (XEN) [VT-D]dmar.c:272: found endpoint: bdf = 0:1d.2
>
> (XEN) [VT-D]dmar.c:272: found endpoint: bdf = 0:1d.7
>
> (XEN) [VT-D]dmar.c:272: found endpoint: bdf = 0:1a.0
>
> (XEN) [VT-D]dmar.c:272: found endpoint: bdf = 0:1a.1
>
> (XEN) [VT-D]dmar.c:272: found endpoint: bdf = 0:1a.2
>
> (XEN) [VT-D]dmar.c:272: found endpoint: bdf = 0:1a.7
>
> (XEN) [VT-D]dmar.c:460: found ACPI_DMAR_RMRR
>
> (XEN) [VT-D]dmar.c:359: ED_DEBUG: RMRR base:0x0000000000000000  RMRR
> end:0x0000000000000000
>
> (XEN) [VT-D]dmar.c:363: RMRR is incorrect.
>
> (XEN) Failed to parse ACPI DMAR.  Disabling VT-d.
>
>
>
> ________________________________
>
> From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
> [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Han, Weidong
> Sent: Wednesday, August 06, 2008 2:59 AM
> To: Kumar, Venkat; Cui, Dexuan; Bryan York
> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
> Subject: RE: [Xen-devel] Xen-3.2.1 VT-d Support (NOT SURE WHETHER IT'S
> ABUGOR...)
>
>
>
> Venkat,
>
>
>
> I think you are using PEG (which means IGD is disabled), right? This is a
> BIOS. I suggest you use IGD at this moment.
>
>
>
> Randy (Weidong)
>
>
>
> ________________________________
>
> From: Kumar, Venkat [mailto:Venkat.Kumar@xxxxxxx]
> Sent: 2008年8月6日 16:22
> To: Cui, Dexuan; Bryan York; Han, Weidong
> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
> Subject: RE: [Xen-devel] Xen-3.2.1 VT-d Support (NOT SURE WHETHER IT'S
> ABUGOR...)
>
> As Dexuan suggested I used xen3.2.2-rc2.
>
> With this version of xen with vtd=1 boot parameter I could boot successfully
> (I was not able to boot with vtd=1 in xen-3.2.1).
>
> But I see the following in the boot messages when I do a xm dmesg after a
> boot.
>
> ===================================================
>
> (XEN) Xen heap: 14MB (14956kB)
>
> (XEN) Domain heap initialised: DMA width 32 bits
>
> (XEN) Processor #0 6:15 APIC version 20
>
> (XEN) Processor #1 6:15 APIC version 20
>
> (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
>
> (XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
>
> (XEN) [VT-D]dmar.c:441: RMRR is incorrect.
>
> (XEN) Failed to parse ACPI DMAR.  Disabling VT-d.
>
> (XEN) [VT-D]ACPI DMAR:No DMAR devices found
>
> ===================================================
>
> If you observe the messages highlighted in RED.
>
> VT-d is getting disabled due to some RMRR incorrect problem and in the
> dmar.c code this is the condition which is causing this.
>
>
>
>     if ( rmrr->base_address >= rmrr->end_address )
>
>     {
>
>         dprintk(XENLOG_ERR VTDPREFIX, "RMRR is incorrect.\n");
>
>         return -EFAULT;
>
>     }
>
> This code is in acpi_parse_one_rmrr function.
>
>
>
> Thx,
>
> Venkat
>
> ================================
>
> Venkata Kumar Duvvuru,
>
> LSI Engenio,
>
> Adv. Development,
>
> Bangalore.
>
> Mob: +91-9880318542
>
> Off : +91-80-41978700 ( Extn : 3544 )
>
> ================================
>
> ________________________________
>
> From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
> [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Cui, Dexuan
> Sent: Wednesday, August 06, 2008 10:58 AM
> To: Bryan York; Han, Weidong
> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
> Subject: RE: [Xen-devel] Xen-3.2.1 VT-d Support (NOT SURE WHETHER IT'S A
> BUGOR...)
>
>
>
> Hi Bryan,
>
> According to our tests using the latest xen-unstable, the assignment of SATA
> controller (like in AHCI mode or Enhanced mode) should be OK on the DQ35
> platform.
>
>
>
> For xen-unstable, I guess you didn't specify "iommu=1" in the Xen grub
> entry? By default, VT-d is not enabled in the latest xen-unstable.
>
>
>
> Thanks,
> -- Dexuan
>
>
>
>
>
> ________________________________
>
> From: Bryan York [mailto:bryan.york@xxxxxxxxx]
> Sent: 2008年8月6日 12:10
> To: Han, Weidong
> Cc: Cui, Dexuan; xen-devel@xxxxxxxxxxxxxxxxxxx
> Subject: Re: [Xen-devel] Xen-3.2.1 VT-d Support (NOT SURE WHETHER IT'S A
> BUGOR...)
>
> Hello,
>
> Thanks for the information. I'm trying to forward a PCI express SATA/PATA
> controller with a JMicron chipset. (Info here.) I know there have been
> problems with graphics cards in general, but I thought it was possible with
> disk controllers? Is there a list of well supported hardware (disk
> controllers) that can be used for VT-d?
>
> Also, your RMRR explanation makes sense for the mercurial 3.2-testing, but
> what about xen-unstable? I didn't see any errors there besides "I/O
> Virtualisation disabled" with no other reason listed.
>
> Thanks so much for your help on this. I will also try removing the disk
> controller and see if the mercurial Xen-3.2-testing boots without RMRR
> errors.
>
> Regards,
>
> -Bryan
>
> On Tue, Aug 5, 2008 at 6:27 PM, Han, Weidong <weidong.han@xxxxxxxxx> wrote:
>
> Hi Bryan,
>
>
>
> I think you are using an add-on graphic card (iGfx disabled), right? We also
> met this issue before. Obviously it's a BIOS issue (RMRR is incorrect). When
> parsing apci dmar failed, VT-d will be disabled.
>
>
>
> Randy (Weidong)
>
>
>
> ________________________________
>
> From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
> [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Bryan York
> Sent: 2008年8月6日 6:17
> To: Cui, Dexuan
> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
>
> Subject: RE: [Xen-devel] Xen-3.2.1 VT-d Support (NOT SURE WHETHER IT'S A
> BUGOR...)
>
>
>
> Hello,
>
> I'm also seeing this exact same problem. I've posted on xen-users, but did
> not get an answer. Venkat seems to be experiencing the same problem as me. I
> have the latest BIOS for my motherboard DQ35JO. (BIOS ver 933) I'm running a
> Q6600 as well. It hangs at "Brought up 4 CPUs." I've tried all sorts of
> combinations of Xen versions and kernels to get this working. And, yes, VT
> and VT-d are enabled in my BIOS.
>
> 1) CentOS 5.2 SRPMS compiled with Xen 3.2.1. Adding pci=nommconf does not
> get it to boot.
>
> 2) Mercurial Xen-3.2-testing. Here, after adding pci=nommconf, I get:
>
> (XEN) [VT-D]dmar.c:441: RMRR is incorrect.
> (XEN) Failed to parse ACPI DMAR.  Disabling VT-d.
> (XEN) [VT-D]ACPI DMAR:No DMAR devices found
>
> 3) Mercurial Xen-unstable, I get "I/O Virtualisation disabled" I can't find
> any reason why this is the case. VT-D is not enabled.
>
> I would really like to find the cause for this. I don't know if this is a
> BIOS bug or what. This board is supposedly supported on Xen per the VT-D
> wiki. Other people seem to have the same problem as me:
>
> http://forums12.itrc.hp.com/service/forums/bizsupport/questionanswer.do?admit=109447627+1217898320366+28353475&threadId=1234588
>
> Thanks for any and all help,
>
> -Bryan
>
> The BIOS version I use may be not the exact one as yours. I mean I expect
> you're
> not using a very old BIOS. :-)
>
> How about trying the xen 3.2.2-rc2 and the latest xen-unstable first? There
> are some VT-d bug fixes between 3.2.1 (you're using it) and 3.2.2-rc2.
>
> Thanks,
>
> -- Dexuan
>
> ________________________________
>
> From: Kumar, Venkat [mailto:Venkat.Kumar@xxxxxxx] Sent: 2008年8月5日 14:37 To:
> Cui, Dexuan; xen-devel@xxxxxxxxxxxxxxxxxxx Subject: RE: [Xen-devel]
> Xen-3.2.1 VT-d Support (NOT SURE WHETHER IT'S A BUG
> OR...)
>
> If you don't mind can you send me the BIOS version you are using?
>
> Thx,
>
> Venkat
>
> ================================
>
> Venkata Kumar Duvvuru,
>
> LSI Engenio,
>
> Adv. Development,
>
> Bangalore.
>
> Mob: +91-9880318542
>
> Off : +91-80-41978700 ( Extn : 3544 )
>
> ================================
>
> ________________________________
>
> From: Cui, Dexuan [mailto:dexuan.cui@xxxxxxxxx] Sent: Tuesday, August 05,
> 2008 12:03 PM To: Kumar, Venkat; xen-devel@xxxxxxxxxxxxxxxxxxx Subject: RE:
> [Xen-devel] Xen-3.2.1 VT-d Support (NOT SURE WHETHER IT'S A BUG
> OR...)
>
> Hi Venkat,
>
> Can you confirm you're using the latest BIOS? Can you try the xen 3.2.2-rc2
> and
> the latest xen-unstable on the same host?
>
> I have the similar host. Xen 3.2.2-rc2 and the latest xen-unstable both work
> well on it.
>
> Thanks,
>
> -- Dexuan
>
> ________________________________
>
> From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
> [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Kumar, Venkat
> Sent: 2008年8月5日 12:21 To: xen-devel@xxxxxxxxxxxxxxxxxxx Subject: [Xen-devel]
> Xen-3.2.1 VT-d Support (NOT SURE WHETHER IT'S A BUG OR...)
>
> After building Xen-3.2.1 I could successfully boot into it if I avoid giving
> vtd=1 as the boot parameter.
>
> If I pass vtd=1 as the boot parameter to xen, the system hangs while
> booting.
>
> The system hangs after the following messages
>
> ============================
>
> Boot messages
>
> [XEN]…..
>
> [Xen] Brought up 2 CPUs...
>
> =========================
>
> My chipset DQ35 series is having VT-d support and is enabled in the BIOS as
> well.
>
> My domain0 is CentOS-5.2(2.6.18.8-xen).
>
> Is this a possible bug or some thing else??
>
> Any Idea??
>
> Thx,
>
> Venkat
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel
>
>

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel