Re: [Xen-devel] [PATCH] xen/x86: Force removal of memory range when not covered by MTRRs

[Stefan Bader <stefan.bader@xxxxxxxxxxxxx>]

On 15.02.2013 10:19, Jan Beulich wrote:
>>>> On 14.02.13 at 18:11, Stefan Bader <stefan.bader@xxxxxxxxxxxxx> wrote:
>> On a machine that could not cover all its RAM with MTRR ranges,
>> a crash on boot as dom0 was caused by dom0 trying to create
>> kernel mapping tables for the clipped range.
>>
>> (XEN) WARNING: MTRRs do not cover all of memory.
>> (XEN) Truncating RAM from 9109504kB to 9043968kB
>> ...
>> (XEN)  0000000228000000 - 000000022c000000 (unusable)
>> ...
>> [    0.000000] init_memory_mapping: 0000000228000000-000000022c000000
>> [    0.000000]  0228000000 - 022c000000 page 4k
>> [    0.000000] kernel direct mapping tables up to 22c000000 @
>>                1e97d8000-1e97fa000
>> (XEN) mm.c:825:d0 Non-privileged (0) attempt to map I/O space 00228000
>> (XEN) mm.c:1222:d0 Failure in alloc_l1_table: entry 0
>> (XEN) mm.c:2177:d0 Error while validating mfn 81de (pfn 1e97d8) for
>>       type 1000000000000000: caf=8000000000000003 taf=1000000000000001
>> (XEN) mm.c:2985:d0 Error while pinning mfn 81de
>>
>> Setting the range in E820 to E280_UNUSABLE seems ambigous as
>> this is the same type that gets used for memory to be used only
>> as guest memory (using dom0_mem=).
> 
> Since when is E820_UNUSABLE memory being used as guest
> memory? If that's indeed the case, that's the bug to fix. The
> above data to me shows, however, that the range above
> 228000000 is considered invalid. So then the question is why the
> kernel tries to map that memory in the first place (the hypervisor
> rejecting the attempt, despite Dom0 being privileged, seems
> correct to me, as the range is also known not to be MMIO).

That seems to be done for a while now... On my testbox, when using dom0_mem=:

(XEN)  00000000dfe9e000 - 00000000dfea0000 type 9
(XEN)  00000000dfea0000 - 00000000dfeb2000 (ACPI data)
(XEN)  00000000dfeb2000 - 00000000dfee0000 (ACPI NVS)
(XEN)  00000000dfee0000 - 00000000f0000000 (reserved)
(XEN)  00000000ffe00000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 0000000820000000 (usable)

In dom0:
[    0.000000] Xen: [mem 0x00000000dfe9e000-0x00000000dfe9ffff] type 9
[    0.000000] Xen: [mem 0x00000000dfea0000-0x00000000dfeb1fff] ACPI data
[    0.000000] Xen: [mem 0x00000000dfeb2000-0x00000000dfedffff] ACPI NVS
[    0.000000] Xen: [mem 0x00000000dfee0000-0x00000000efffffff] reserved
[    0.000000] Xen: [mem 0x00000000fec00000-0x00000000fec00fff] reserved
[    0.000000] Xen: [mem 0x00000000fec20000-0x00000000fec20fff] reserved
[    0.000000] Xen: [mem 0x00000000fee00000-0x00000000fee00fff] reserved
[    0.000000] Xen: [mem 0x00000000ffe00000-0x00000000ffffffff] reserved
[    0.000000] Xen: [mem 0x0000000100000000-0x00000002e063ffff] usable
[    0.000000] Xen: [mem 0x00000002e0640000-0x000000081fffffff] unusable

[    0.000000] init_memory_mapping: [mem 0x100000000-0x2e063ffff]
[    0.000000]  [mem 0x100000000-0x2e063ffff] page 4k
[    0.000000] kernel direct mapping tables up to 0x2e063ffff @ [mem
0x3f163000-0x4006ffff]

[   10.288549] e820: reserve RAM buffer [mem 0xdfe90000-0xdfffffff]
[   10.288555] e820: reserve RAM buffer [mem 0x2e0640000-0x2e3ffffff]

There the majority of memory appears as E820_UNUSABLE for dom0 and is used for
guests. Ok, in that case not trying to do any kernel mappings...
Hm, not sure whether I should be worried about that first reserve RAM buffer
covering the type 9 (whatever that is... I think it appeared when I upgraded the
BIOS to get IOMMU back after xsa-36) range...

It seems to be arch/x86/xen/setup.c:xen_memory_setup() that changes things to
unusable...

-Stefan
> 
>> To avoid this, the clipped memory should be dropped completely
>> from E820 (as it is done if setting the memory type fails).
>> This is currently restricted to only the case of memory not
>> coverable by MTRRs (which could be tested). Maybe it should
>> be done in other cases, too.
> 
> No, that's wrong. When dropping the range completely, the
> Dom0 kernel then could allocate MMIO resources in that address
> range, and the devices using those MMIO resources then would
> not work.
> 
> Bottom line - I think this needs to be fixed in the kernel.
> 
> Jan

Attachment: signature.asc
Description: OpenPGP digital signature

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel