RE: [Xen-devel] XEN Panic with VT-d support enabled on Xeon E55xxplatform

["Isabelle, Francois" <Francois.Isabelle@xxxxxxxxxxxxxx>]

>> Did you see 1 second delay after above line was printed? Or the system 
got panic immediately following it?

Yes, maybe a little less but it's not immediate for sure.

François Isabelle 


-----Message d'origine-----
De : Zhao, Yu [mailto:yu.zhao@xxxxxxxxx] 
Envoyé : 8 avril 2009 01:55
À : Isabelle, Francois
Cc : Cui, Dexuan; xen-devel@xxxxxxxxxxxxxxxxxxx
Objet : Re: [Xen-devel] XEN Panic with VT-d support enabled on Xeon 
E55xxplatform

Francois,

Thank you for the log.

(XEN) [VT-D]iommu.c:1237:d32767 domain_context_mapping:PCI: bdf = 0:1d.0

Did you see 1 second delay after above line was printed? Or the system 
got panic immediately following it?

(XEN) DMAR_IQA_REG = 7f79d000
(XEN) DMAR_IQH_REG = 0
(XEN) DMAR_IQT_REG = 20
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) queue invalidate wait descriptor was not executed
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...


Isabelle, Francois wrote:
>>> When you only use iommu=no-intremap and see the panic, can you attached 
>>> >>the xen log?
>>> And can you also try iommu=passthrough,no-intremap and attach the log?
> 
> See attached files.
> 
>>> VT-d Context Entry is used, the DMA requests related to the Context Entry 
>>> can bypass DMA remapping. This can be faster than DMA remapping.
> The BIOS does not have an option for this on my board. But I did mistook it 
> for DMA remapping.
> 
> 
> And thank you for answering my questions.
> 
> François Isabelle
> 
> 
> 
> 
> -----Message d'origine-----
> De : Cui, Dexuan [mailto:dexuan.cui@xxxxxxxxx]
> Envoyé : 7 avril 2009 10:22
> À : Isabelle, Francois; Zhao, Yu
> Cc : xen-devel@xxxxxxxxxxxxxxxxxxx
> Objet : RE: [Xen-devel] XEN Panic with VT-d support enabled on Xeon 
> E55xxplatform
> 
>>>> iommu=xxx
>> Dexuan, 'iommu=no-intremap' is not enough to make it run, ' 
>> iommu=no-qinval,no-intremap' allows booting.
> When you only use iommu=no-intremap and see the panic, can you attached the 
> xen log?
> And can you also try iommu=passthrough,no-intremap and attach the log?
> 
>>> (XEN) [VT-D]dmar.c:485: Host address width 39
>> Shouldn't this be 38 ?
> The Host Address Width (HAW) of the platform is computed as (N+1), where N is 
> the value reported in the Host Address Width field of DMA Remapping Reporting 
> Structure.
> Please see acpi_parse_dmar() or VT-d spec for details.
> 
>>> (XEN) Intel VT-d Snoop Control supported.
>>> (XEN) Intel VT-d DMA Passthrough not supported.
>> Shouldn't that be enabled to support DMA from domU?
> If the "DMA Passthrough" (please don't mistake it for DMA remapping:-) of 
> VT-d Context Entry is used, the DMA requests related to the Context Entry can 
> bypass DMA remapping. This can be faster than DMA remapping. If you host 
> supports DMA passthrough (you could check your BIOS setup menu), in Xen we 
> can enable DMA passthrough by iommu=passthrough (Note, even with this 
> parameter, we only enable passthrough for Dom0 considering secrity)
> 
> 
>>> (XEN) HPET broadcast init failed, turn to PIT broadcast.
>> Does that mean the HPET is not functional?
>>From hpet_fsb_cap_lookup(), looks your HPET timers may work, but they don't 
>>support FSB routing, so won't be used in power management code.
> 
>> Can we expect a good level of performance with interrupt remapping disabled? 
>> Will a domU be able to load a 82576 driver and use the NIC at all?
> I think we should have almost the same performance with Interrupt Remapping 
> enabled or disabled.
> 82576 should work in DomU.
> 
> Thanks,
> -- Dexuan
> 
> 
> 
> -----Original Message-----
> From: Isabelle, Francois [mailto:Francois.Isabelle@xxxxxxxxxxxxxx]
> Sent: Tuesday, April 07, 2009 9:12 PM
> To: Cui, Dexuan; Zhao, Yu
> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
> Subject: RE: [Xen-devel] XEN Panic with VT-d support enabled on Xeon 
> E55xxplatform
> 
> Hi.
> 
> Here is what you both asked for:
> 
>>> Do you have this problem all the times? Can you please grab some logs
>>> and send them to me?
> 
> Yes Yu, all the times. I will package the logs as attachment to this post.
> 
>>> iommu=xxx
> Dexuan, 'iommu=no-intremap' is not enough to make it run, ' 
> iommu=no-qinval,no-intremap' allows booting.
> 
> Here are some parts of the log I don't understand...
> 
>> (XEN) [VT-D]dmar.c:485: Host address width 39
>  Shouldn't this be 38 ?
> 
>> (XEN) Intel VT-d Snoop Control supported.
>> (XEN) Intel VT-d DMA Passthrough not supported.
>  Shouldn't that be enabled to support DMA from domU?
> 
>> (XEN) Intel VT-d Queued Invalidation supported.
>> (XEN) Intel VT-d Interrupt Remapping supported.
> 
>> (XEN) HPET broadcast init failed, turn to PIT broadcast.
>  Does that mean the HPET is not functional?
> 
> Can we expect a good level of performance with interrupt remapping disabled? 
> Will a domU be able to load a 82576 driver and use the NIC at all?
> 
> Thank you both for your replies.
> 
> François Isabelle
> 
> -----Message d'origine-----
> De : Cui, Dexuan [mailto:dexuan.cui@xxxxxxxxx]
> Envoyé : 7 avril 2009 03:05
> À : Zhao, Yu; Isabelle, Francois
> Cc : xen-devel@xxxxxxxxxxxxxxxxxxx
> Objet : RE: [Xen-devel] XEN Panic with VT-d support enabled on Xeon 
> E55xxplatform
> 
>>> (XEN) DMAR_IQA_REG = 7f79d000
>>> (XEN) DMAR_IQH_REG = 0
>>> (XEN) DMAR_IQT_REG = 20
>> Here  the IQT is such a big number 0x20... this seems pretty strange.
> Oh, actually this 0x20 is normal -- actually QT = IQT_REG >> 4 accoriding the 
> format of IQT_REG in VT-d spec. So QT = 0x2 -- this is ok.
> So looks the panic somehow happens on the first invalidation: 
> enable_intremap() -> iommu_flush_iec_global(). I expect your host can boot 
> fine with iommu=no-intremap -- if this still doesn't work, could you please 
> try iommu=no-qinval,no-intremap ?
> 
> Thanks,
> -- Dexuan
> 
> 
> 
> -----Original Message-----
> From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx 
> [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Cui, Dexuan
> Sent: Tuesday, April 07, 2009 12:40 PM
> To: Zhao, Yu; Isabelle, Francois
> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
> Subject: RE: [Xen-devel] XEN Panic with VT-d support enabled on Xeon E55xx 
> platform
> 
>> (XEN) DMAR_IQA_REG = 7f79d000
>> (XEN) DMAR_IQH_REG = 0
>> (XEN) DMAR_IQT_REG = 20
> Here  the IQT is such a big number 0x20... this seems pretty strange.
> 
> Hi François,
> And, could you please try the xen parameter "iommu=no-intremap" ? -- If you 
> still see the panic, please  also add the parameter "nosmp" to see if  there 
> would any change.
> We don't have the same host at hand. :-(  So we need try these on your host.
> BTW, can you boot the same host using the latest native linux-2.6.29-rc8 when 
> you iommu_intel=on and interrupt remapping is enabled?
> 
> PS, what's your BIOS version and date -- could you try to find the latest 
> BIOS for the host? I personally tend to think this is a BIOS issue.
> 
> Thanks,
> -- Dexuan
> 
> 
> 
> -----Original Message-----
> From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx 
> [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Zhao, Yu
> Sent: Tuesday, April 07, 2009 10:31 AM
> To: Isabelle, Francois
> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
> Subject: Re: [Xen-devel] XEN Panic with VT-d support enabled on Xeon E55xx 
> platform
> 
> Hi Francois,
> 
> Do you have this problem all the times? Can you please grab some logs
> and send them to me? The dmesg from a native kernel if you can't boot up
> any version of Xen, and the whole Xen hypervisor log would help us to
> figure out the problem.
> 
> Thanks,
> Yu
> 
> 
> Isabelle, Francois wrote:
>> Hi all,
>>
>> I'm working with a board on which XEN panics with the following message (see 
>> below). From what I can see on the lists, most of the problems on these 
>> platforms seem to be related to broken DMAR tables and RMRR, but this 
>> platform passes the VT-d firmware toolkit 
>> (http://edc.intel.com/Platforms/Xeon-5500/) successfully so I'm tented to 
>> blame XEN for it... should I file a bug entry?
>>
>> >From what I can see in the 'queue invalidation' code, a timeout is reached. 
>> >As anyone seen this on a similar platform?
>>
>>
>> (XEN) Xen version 3.4-unstable (root@localdomain) (gcc version 4.1.2 
>> 20080704 (Red Hat 4.1.2-44)) Fri Apr  3 14:20:00 EDT 2009
>> (XEN) Latest ChangeSet: Mon Mar 30 16:48:26 2009 +0100 19433:d5ddc782bc49
>> (XEN) Command line: console=com1 com1=115200,8n1  iommu=1 loglvl=all 
>> loglvl_guest=all ro
>>
>> ..
>> (XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
>> (XEN) ACPI: HPET id: 0xffffffff base: 0xfed00000
>> (XEN) [VT-D]dmar.c:485: Host address width 39
>> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
>> (XEN) [VT-D]dmar.c:349: dmaru->address = fbffe000
>> (XEN) [VT-D]dmar.c:306: found IOAPIC: bdf = f0:1f.7
>> (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL
>> (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR
>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0
>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1
>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2
>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7
>> (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR
>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0
>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1
>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2
>> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7
>> (XEN) [VT-D]dmar.c:502: found ACPI_DMAR_ATSR
>> (XEN) [VT-D]dmar.c:287: found bridge: bdf = 0:3.0  sec = 5  sub = 5
>> (XEN) [VT-D]dmar.c:287: found bridge: bdf = 0:4.0  sec = 6  sub = e
>> (XEN) [VT-D]dmar.c:287: found bridge: bdf = 0:5.0  sec = 3  sub = 3
>> (XEN) [VT-D]dmar.c:287: found bridge: bdf = 0:7.0  sec = 18  sub = 18
>> (XEN) [VT-D]dmar.c:287: found bridge: bdf = 0:8.0  sec = f  sub = 17
>> (XEN) [VT-D]dmar.c:287: found bridge: bdf = 0:9.0  sec = 4  sub = 4
>> (XEN) [VT-D]dmar.c:287: found bridge: bdf = 0:a.0  sec = 2  sub = 2
>> ..
>> (XEN) Intel VT-d Snoop Control supported.
>> (XEN) Intel VT-d DMA Passthrough not supported.
>> (XEN) Intel VT-d Queued Invalidation supported.
>> (XEN) Intel VT-d Interrupt Remapping supported.
>> (XEN) DMAR_IQA_REG = 7f79d000
>> (XEN) DMAR_IQH_REG = 0
>> (XEN) DMAR_IQT_REG = 20
>> (XEN)
>> (XEN) ****************************************
>> (XEN) Panic on CPU 0:
>> (XEN) queue invalidate wait descriptor was not executed
>> (XEN) ****************************************
>> (XEN)
>> (XEN) Reboot in five seconds...
>>
>> François Isabelle
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@xxxxxxxxxxxxxxxxxxx
>> http://lists.xensource.com/xen-devel
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel
> 


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel