Re: [Xen-devel] Is: MTRR issue + Xen + memory clipping? Was:Re: Xen not working with stock Debian Wheezy 3.2 kernel on a Core 2 Duo box

[Stefan Bader <stefan.bader@xxxxxxxxxxxxx>]

On 08.07.2013 21:47, Konrad Rzeszutek Wilk wrote:
> On Mon, Jul 08, 2013 at 02:16:07PM +0100, Jan Beulich wrote:
>>>>> On 08.07.13 at 14:50, Wei Liu <wei.liu2@xxxxxxxxxx> wrote:
>>> On Mon, Jul 08, 2013 at 12:29:02PM +0100, Jan Beulich wrote:
>>>> One question of course is where this pretty unusual "unusable"
>>>> memory block comes from on that system. Is this block visible the
>>>> same way when booting a native kernel, or is this being forced to
>>>> "unusable" by Xen?
>>>
>>> Vanilla 3.10 + Xen unstable:
>>> [    0.000000] Xen: [mem 0x0000000000000000-0x000000000008efff] usable
>>> [    0.000000] Xen: [mem 0x000000000008f000-0x00000000000fffff] reserved
>>> [    0.000000] Xen: [mem 0x0000000000100000-0x00000000ce699fff] usable
>>> [    0.000000] Xen: [mem 0x00000000ce69a000-0x00000000ce6f0fff] ACPI NVS
>>> [    0.000000] Xen: [mem 0x00000000ce6f1000-0x00000000cf5fafff] usable
>>> [    0.000000] Xen: [mem 0x00000000cf5fb000-0x00000000cf607fff] reserved
>>> [    0.000000] Xen: [mem 0x00000000cf608000-0x00000000cf6a4fff] usable
>>> [    0.000000] Xen: [mem 0x00000000cf6a5000-0x00000000cf6a9fff] ACPI data
>>> [    0.000000] Xen: [mem 0x00000000cf6aa000-0x00000000cf6aafff] usable
>>> [    0.000000] Xen: [mem 0x00000000cf6ab000-0x00000000cf6f1fff] ACPI NVS
>>> [    0.000000] Xen: [mem 0x00000000cf6f2000-0x00000000cf6fefff] ACPI data
>>> [    0.000000] Xen: [mem 0x00000000cf6ff000-0x00000000cf6fffff] usable
>>> [    0.000000] Xen: [mem 0x00000000cf700000-0x00000000cfffffff] reserved
>>> [    0.000000] Xen: [mem 0x00000000fee00000-0x00000000fee00fff] reserved
>>> [    0.000000] Xen: [mem 0x00000000fff00000-0x00000000ffffffff] reserved
>>> [    0.000000] Xen: [mem 0x0000000100000000-0x0000000227ffffff] usable
>>> [    0.000000] Xen: [mem 0x0000000228000000-0x000000022bffffff] unusable
>>> [    0.000000] ERROR: earlyprintk= xenboot already used
>>> [    0.000000] NX (Execute Disable) protection: active
>>> [    0.000000] SMBIOS 2.4 present.
>>> [    0.000000] No AGP bridge found
>>> [    0.000000] e820: last_pfn = 0x228000 max_arch_pfn = 0x400000000
>>> [    0.000000] e820: last_pfn = 0xcf700 max_arch_pfn = 0x400000000
>>> [    0.000000] Scanning 1 areas for low memory corruption
>>> [    0.000000] init_memory_mapping: [mem 0x00000000-0x000fffff]
>>> [    0.000000] init_memory_mapping: [mem 0x219e00000-0x219ffffff]
>>> [    0.000000] init_memory_mapping: [mem 0x218000000-0x219dfffff]
>>> [    0.000000] init_memory_mapping: [mem 0x200000000-0x217ffffff]
>>> [    0.000000] init_memory_mapping: [mem 0x00100000-0xce699fff]
>>> [    0.000000] init_memory_mapping: [mem 0xce6f1000-0xcf5fafff]
>>> [    0.000000] init_memory_mapping: [mem 0xcf608000-0xcf6a4fff]
>>> [    0.000000] init_memory_mapping: [mem 0xcf6aa000-0xcf6aafff]
>>> [    0.000000] init_memory_mapping: [mem 0xcf6ff000-0xcf6fffff]
>>> [    0.000000] init_memory_mapping: [mem 0x100000000-0x1ffffffff]
>>> [    0.000000] init_memory_mapping: [mem 0x21a000000-0x227ffffff]
>>>
>>> We also have the similar unusable block, however the kernel doesn't map it.
>>
>> Right, iirc a fix for this was done not too long ago. Konrad may
>> recall further details...
> 
> Stefan hit this I think. With the MTRR clipping blowing things. CC-ing him
> here.

Not personally but we had a bug report about this. Unfortunately it was a bit
confusing as in the beginning it was not really obvious whether the problem was
or was not fixed in later kernels or whether the different installations used
different dom0_mem arguments.

Reading the old bug report (which never completed as it seemed the reporters
apparently lost interest) I think the problem was that the hypervisor detects
the MTRR problem and set the remainder of memory as unusable.
Then dom0 kernel (and if I parse that report right, it may have been fixed
between 3.2 and 3.3 but hard to say when all you get is them saying yes or no)
would somehow still try to put mappings in there.

The link below is that bug report. Not sure of how much help it is.


[1] https://bugs.launchpad.net/ubuntu/+source/xen/+bug/1111470
>>
>>> 3.2.0-4-amd64 (just found out that there's actually backtrace in dmesg):
>>> [    0.000000] BIOS-provided physical RAM map:
>>> [    0.000000]  BIOS-e820: 0000000000000000 - 000000000008f000 (usable)
>>> [    0.000000]  BIOS-e820: 000000000008f000 - 00000000000a0000 (reserved)
>>> [    0.000000]  BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
>>> [    0.000000]  BIOS-e820: 0000000000100000 - 00000000ce69a000 (usable)
>>> [    0.000000]  BIOS-e820: 00000000ce69a000 - 00000000ce6f1000 (ACPI NVS)
>>> [    0.000000]  BIOS-e820: 00000000ce6f1000 - 00000000cf5fb000 (usable)
>>> [    0.000000]  BIOS-e820: 00000000cf5fb000 - 00000000cf608000 (reserved)
>>> [    0.000000]  BIOS-e820: 00000000cf608000 - 00000000cf6a5000 (usable)
>>> [    0.000000]  BIOS-e820: 00000000cf6a5000 - 00000000cf6aa000 (ACPI data)
>>> [    0.000000]  BIOS-e820: 00000000cf6aa000 - 00000000cf6ab000 (usable)
>>> [    0.000000]  BIOS-e820: 00000000cf6ab000 - 00000000cf6f2000 (ACPI NVS)
>>> [    0.000000]  BIOS-e820: 00000000cf6f2000 - 00000000cf6ff000 (ACPI data)
>>> [    0.000000]  BIOS-e820: 00000000cf6ff000 - 00000000cf700000 (usable)
>>> [    0.000000]  BIOS-e820: 00000000cf700000 - 00000000d0000000 (reserved)
>>> [    0.000000]  BIOS-e820: 00000000fff00000 - 0000000100000000 (reserved)
>>> [    0.000000]  BIOS-e820: 0000000100000000 - 000000022c000000 (usable)
>>
>> So no such block right off the BIOS. You're not using Xen TXT code
>> by chance? Off the top of my head I don't recall any other place
>> where multiple RAM pages might get turned into "unusable".
>>
>> Jan
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@xxxxxxxxxxxxx
>> http://lists.xen.org/xen-devel

Attachment: signature.asc
Description: OpenPGP digital signature

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel