Re: [Xen-devel] [xen-unstable] Commit 2ca9fbd739b8a72b16dd790d0fff7b75f5488fb8 AMD IOMMU: allocate IRTE entries instead of using a static mapping, makes dom0 boot process stall several times.

[Sander Eikelenboom <linux@xxxxxxxxxxxxxx>]

Tuesday, August 6, 2013, 5:47:04 PM, you wrote:

> On 8/5/2013 2:48 AM, Jan Beulich wrote:
>>>>> On 22.07.13 at 22:50, Sander Eikelenboom <linux@xxxxxxxxxxxxxx> wrote:
>>> After commit 2ca9fbd739b8a72b16dd790d0fff7b75f5488fb8 AMD IOMMU: allocate
>>> IRTE entries instead of using a static mapping, booting dom0 stalls several
>>> times.
>>> Sometimes this results in RCU stall warnings from the dom0 kernel, hitting
>>> the "any" key, on normal or serial console, makes the boot continue for a
>>> while but it stalls several times.
>>> (It also stalls on shutdown BTW)
>>>
>>> I have bisected it to this commit running kernel 3.10.2 and Debian wheezy.
>>>
>>> xl-dmesg and dmesg from a boot with stall are attached, snippet from dmesg
>>> with RCU stall appended below.
>>>
>>> Note that i do have to run with iommu='no-amd-iommu-perdev-intremap' due to 
>>> the
>>> bios bug that probably will never get fixed.
>>> When i run without the 'no-amd-iommu-perdev-intremap' the iommu is disabled 
>>> and
>>> stalls do not occur.
>> I can only hope that someone with access to a system exposing this
>> problem (you?) can debug it - I certainly have no system where this
>> issue could be seen, and I don't recall Suravee having noticed
>> anything like this in his testing of those patches.
>>
>> Jan

> I don't have the system which has the exact same BIOS IVRS IOAPIC bug as 
> mentioned
> in the origin xl dmesg output you sent. In your case the special IVHD entry 
> for the
> first IOAPIC with ID 0x6 is missing from IVRS table.

> (XEN) AMD-Vi: IVHD Special: 0000:00:14.0 variety 0x2 handle 0
> (XEN) AMD-Vi: IVHD Device Entry: type 0x48 id 0 flags 0
> (XEN) AMD-Vi: IVHD Special: 0000:00:00.1 variety 0x1 handle 0x7
> (XEN) IVHD Error: no information for IO-APIC 0x6
> (XEN) vmap(ffff82c000802000:0x4)
> (XEN) AMD-Vi: IOMMU 0 Enabled.
> (XEN) AMD-Vi: Using global interrupt remap table is not recommended (see 
> XSA-36)!
> (XEN) I/O virtualisation enabled

> Here, I was expecting to see a print out showing

> (XEN) AMD-Vi: IVHD Special: 0000:00:14.0 variety 0x1 handle 0x6

> I could imagine this would cause IOMMU to not correctly setup, and probably 
> should
> be disabled.

Well apart from the dom0 boot stalls, everthing is working fine after the 
system is booted, including the passthrough of 4 pci(e) devices.
From closer inspection it seems to be when enabling the sata controller and the 
NICs during dom0 boot.
The question is why it stalls, and why it seems to resume when pressing any key 
on the console.
(so unfortunately trying to use xen debug keys to find out what it's occupied 
with doesn't work in this case)

The Bisection pointed to that changeset, which is quite large, i will see if i 
can get some more info.

--
Sander

> Suravee





_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel