Re: [Xen-users] Xen Hypervisor memory utilization (leak?)

[Pasi Kärkkäinen <pasik@xxxxxx>]

On Thu, Mar 18, 2010 at 08:28:05AM -0500, Frank S Fejes III wrote:
> Hello, everybody.  Over the past couple months I've noticed that a few
> of our Xen servers are reporting (via xentop) hypervisor memory
> utilization discrepancies between the sum of VM memory allocations and
> the actual memory used.  In time, the free memory drops to a point
> where no new VMs can be started.  Below is sample xentop output from
> yesterday with extra columns chopped.  The hypervisor reports that it
> is using 32gb of ram, however the sum of the running VMs is 16gb.
> Ballooning is disabled and dom0_mem is set to 4gb.
> 
> xentop - 15:38:20   Xen 3.4.1_19718_04-2.1
> 12 domains: 2 running, 10 blocked, 0 paused, 0 crashed, 0 dying, 0 shutdown
> Mem: 33549248k total, 33241552k used, 307696k free    CPUs: 4 @ 2992MHz
>       NAME  STATE   CPU(sec) CPU(%)     MEM(k) MEM(%)  MAXMEM(k) MAXMEM(%)
> xxxxxxxxxx --b---       1584    0.2     547840    1.6     547840       1.6
>   Domain-0 -----r     165894   34.3    4192768   12.5   no limit       n/a
> xxxxxxxxxx --b---      15166   22.3    3071916    9.2    3076096       9.2
> xxxxxxxxxx -----r       1328   77.8    1028012    3.1    1028096       3.1
> xxxxxxxxxx --b---      55433    4.7    1027880    3.1    1028096       3.1
> xxxxxxxxxx --b---      67527    4.8    1027880    3.1    1028096       3.1
> xxxxxxxxxx --b---      73019    6.8    1027880    3.1    1028096       3.1
> xxxxxxxxxx --b---      61266    6.8    1027880    3.1    1028096       3.1
> xxxxxxxxxx --b---      78880    6.2    1028012    3.1    1028096       3.1
> xxxxxxxxxx --b---        969    7.7     516012    1.5     516096       1.5
> xxxxxxxxxx --b---      10544    1.5    1027880    3.1    1028096       3.1
> xxxxxxxxxx --b---      21001    5.8    1027880    3.1    1028096       3.1
> 
> Our dom0 is opensuse 11.2 and, as you can see above, our Xen version
> is 3.4.1 which is provided by the opensuse project.  This behavior is
> only seen on our "development" Xen servers where we routinely
> create/clone/destroy a large number of (mostly HVM) VMs, however when
> doing any of those functions I cannot reproduce the "leak".  That is
> to say, when I create a VM the hypervisor memory free statistic drops
> in an amount corresponding to the size of the VM.  When the VM is
> destroyed, the free statistic increases back to where it was
> previously.
> 
> Has anyone else seen this behavior before?  I have not been able to
> find any way to get the hypervisor to reclaim some of the lost memory
> and, as a result, I have been forced to reboot the server.  On systems
> with 20+ VMs where we don't have the option for live migration this
> can be rather traumatic.
> 
> Any help or insight would be greatly appreciated!
> 

I think you should report this to Novell bugzilla!

-- Pasi


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users