Re: [Xen-devel] [Xen-users] 32Bit domus and RAM between 128GB and 168 GB

["Jan Beulich" <JBeulich@xxxxxxxx>]

(dropping xen.users, to avoid cross posting)

>>> On 05.01.17 at 12:47, <george.dunlap@xxxxxxxxxx> wrote:
> On Wed, Jan 4, 2017 at 6:00 PM, Austin S. Hemmelgarn
> <ahferroin7@xxxxxxxxx> wrote:
>> On 2017-01-04 09:50, Jan Marquardt wrote:
>>> unfortunately we still have a lot of paravirtual guests with 32 Bit OS
>>> and are currently running in some problems.
>>>
>>> As far as I understand the documentation in xend-config.sxp, if a
>>> physical machine has between 128 GB and 168 GB RAM, 32 Bit guests should
>>> be able to consume up to 168 GB.
>>>
>>> # 32-bit paravirtual domains can only consume physical
>>> # memory below 168GB. On systems with memory beyond that address,
>>> # they'll be confined to memory below 128GB.
>>> # Using total_available_memory (in GB) to specify the amount of memory
>>> reserved
>>> # in the memory pool exclusively for 32-bit paravirtual domains.
>>> # Additionally you should use dom0_mem = <-Value> as a parameter in
>>> # xen kernel to reserve the memory for 32-bit paravirtual domains, default
>>> # is "0" (0GB).
>>>
>>> In our case we have four systems with 160 GB RAM, but the guests are not
>>> able to start if there are already guests running which are consuming
>>> 128 GB RAM in sum.

In the end things depend on memory layout: The factor is not the
amount of RAM in the system, but how all the memory is scattered
across the physical address space. No parts of it must live above
the 168Gb boundary, for memory beyond 128Gb to be available to
32-bit PV guests. Posting a boot log of Xen might help decide what's
going on in the case here.

Jan

>>> Is this a bug or do we miss something?
>>
>> 168GB seems like a really odd number, and I'd be very inclined to believe
>> that this is a typo in the documentation (that is, it should say 'physical
>> memory below 128GB').
> 
> I've just been having a chat with Andy Cooper, and apparently the
> comment is actually not a typo; the explanation is rather complicated
> but the summary is that the actual practical limit for modern 32-bit
> Linux guests is 128 GB anyway.
> 
> I don't think this has been documented well anywhere -- this is
> probably something we should change.
> 
> Jan,  Can you report the exact error message you got?  At a bare
> minimum we should report a useful error message when people encounter
> this situation.
> 
>  -George



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel