Re: [Xen-devel] dom0 not seeing all of the assigned memory

[Sven KÃhler <sven.koehler@xxxxxxxxx>]

Am 19.03.2012 12:09, schrieb David Vrabel:
> On 15/03/12 20:07, Sven KÃhler wrote:
>> Am 15.03.2012 19:54, schrieb David Vrabel:
>>> On 15/03/12 18:33, Konrad Rzeszutek Wilk wrote:
>>>> On Sun, Mar 11, 2012 at 11:20:02PM +0100, Sven KÃhler wrote:
>>>>>
>>>>> I have the same problem. I believe this also happens even if the
>>>>> ballooning driver is disabled in dom0 kernel.
>>>
>>> Well, yes.  The balloon driver needs to be enabled and a new target set
>>> to get back the memory released during boot.
>>
>> The dom0 releases pages and passes them back to the hypervisor?
>> But why does it do so in the first place?
> 
> Because the pages are not accessible. They overlap with BIOS or device
> memory.
> 
> Later kernels are better at releasing more pages than earlier ones and
> so it may appear that the domain ends up with less pages but it really
> has the same number of usable pages.

This all makes sense. But I don't have the feeling, this is a proper
explanation of what I'm seeing.

"xl list" shows that dom0 is still at 511M. If the pages were really
released back to the hypervisor, then "xl list" should show an amount of
memory somewhere near the the value that "free" shows. But this isn't
the case. Was it wrong when I said that the pages are passed back to the
hypervisor?

>>>>>> Make sure you have added the following patch to your Xen if you are
>>>>>> using a recent >=3.1 kernel. Unfortunately it isn't in Xen 4.1.x, but
>>>>>> I guess it should be added to there:
>>>>>> http://xenbits.xen.org/hg/staging/xen-unstable.hg/rev/c56dd5eb0fa2
>>>>>>
>>>>>> See if it helps.
>>>>>
>>>>> I recompiled the hypervisor (this was a patch for the hypervisor,
>>>>> right?) and dom0 only has around 413MB inspite of dom0_mem=512M
>>>>> hypervisor parameter.
>>>
>>> You need dom0_mem=max:512M
>>
>> I can't find documentation on what that does. But if I had to make an
>> educated guess, I would assume that dom0_mem=max:512M sets an upper
>> bound on the memory that dom0 may consume. It sounds like, the
>> hypervisor may take memory away from dom0, so that dom0 drops below the
>> specified 512MB.
> 
> The documentation isn't great. dom0_mem sets two memory related limits:
> a) the initial number of pages; and b) the maximum possible number of
> pages (maximum reservation).

As far as I can see, there are three variables here:
- the initial amount of pages that dom0 gets
- the minimal amount of pages that dom0 should have
- the maximal amount of pages that dom0 can have

Please please fix the documentation!

Rumors about what dom0_mem=X does are spreading. And most pages I found
said that it would statically assign memory to dom0. What you're saying
sounds like dom0 is still allowed to grow if one does only use
dom0_mem=X but not dom0_mem=max:X - which is more dynamic than static.

> Without the 'max:' option, the maximum reservation is unlimited.
> 
> Since 3.0.5, Linux has used the minimum of the maximum reservation and
> the total amount of physical RAM to size its page tables etc.

I tried dom0_mem=max:512M as far as I remember. And it didn't improve
the situation (kernel 3.2.10). Also, when dom0 is booting without xen,
the amount of memory shown by free was not nearly as far away from the
amount of physical ram as when dom0 is booting with xen and dom0_mem=...

Something still smells fishy to me!

Anyway: I hope some of the developers have reproduced the problem and
are about to fix it.


Regards,
  Sven

Attachment: signature.asc
Description: OpenPGP digital signature

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel