RE: [Xen-devel] [PATCH] balloon: selfballooning and post memory info via xenbus,

["Dan Magenheimer" <dan.magenheimer@xxxxxxxxxx>]

(Your reply came to me but not to the list... not sure why.
So I've attached your full reply below.)

> ah ok, that is my failure, I need a bigger swapdisk ;)

Yes, definitely.  If you are creating the swapdisk on an ext3
filesystem, you might try using sparse files.  They won't
take up much disk space unless/until they get swapped-to.
There might be some performance ramifications though.
(My testing has been with swap disk as a logical volume
so I can't try sparse.)

> Ok, our plan is to have a high availbilty xen farm. Now we're 
> beginning
> with 2 Suns X2200, each has 16GB RAM. The idea, why we like to use
> selfballooning, because of peak traffic on a server, normal a server
> needs about 256MB, but when it needs more, it shouldn't be a 
> problem to
> give it 4GB. The idea is not to overbook the memory, but have the
> ability to get rid of memory failures because of peaks.

Exactly what it is intended for!

I'd be interested in how it works for guests with memory=4096
and higher.  All of my testing so far has been on a machine with
only 2GB of physical memory so I can test lots of guests but
no large guests.

Thanks,
Dan

> -----Original Message-----
> From: viets@xxxxxxx [mailto:viets@xxxxxxx]
> Sent: Friday, May 16, 2008 9:49 AM
> To: dan.magenheimer@xxxxxxxxxx; xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
> Subject: Re: [Xen-devel] [PATCH] balloon: selfballooning and 
> post memory
> info via xenbus,
> 
> 
> Dan Magenheimer wrote:
> >> thanks for the patch, I was waiting for this feature.
> >
> > Thanks very much for the testing and feedback!  Could you
> > comment on what you plan to use it for?  (Keir hasn't accepted
> > it yet, so I am looking for user support ;-)
> 
> Ok, our plan is to have a high availbilty xen farm. Now we're 
> beginning
> with 2 Suns X2200, each has 16GB RAM. The idea, why we like to use
> selfballooning, because of peak traffic on a server, normal a server
> needs about 256MB, but when it needs more, it shouldn't be a 
> problem to
> give it 4GB. The idea is not to overbook the memory, but have the
> ability to get rid of memory failures because of peaks.
> 
> >
> > First question: Do you have a swap (virtual) disk configured and,
> > if so, how big is it?  (Use "swapon -s" and the size shows in KB.)
> > Selfballooning shouldn't be run in a domain with no swap disk.
> > Also, how big is your "memory=" in your vm.cfg file?
> 
> #kernel = "/boot/xen-3.2.0/vmlinuz-2.6.18.8-xenU"
> #kernel = "/boot/vmlinuz-2.6.18.8-xenU"
> kernel = "/boot/vmlinuz-selfballooning"
> memory = 256
> maxmem = 8192
> vcpu = 4
> name = "test.work.de"
> vif = [ 'bridge=xenvlan323' ]
> disk = [ 'phy:/dev/sda,hda,w', 'file:/var/swap.img,hdb,w' ]
> root = "/dev/hda ro"
> extra = 'xencons=tty'
> 
> 
> 
> swap_size = 256M
> 
> >
> > I'm not able to reproduce your dd failure at all, even with
> > bs=2047M (dd doesn't permit larger values for bs).
> > Your program (I called it "mallocmem") does eventually fail for
> > me but not until i==88.  However, I have a 2GB swap disk configured.
> 
> ah ok, that is my failure, I need a bigger swapdisk ;)
> >
> > I think both tests are really measuring the total virtual memory
> > space configured, e.g. the sum of physical memory (minus kernel
> > overhead) and configured swap space.  I think you will find that
> > both will fail similarly with ballooning off and even on a physical
> > system, just at different points in virtual memory usage.
> > Indeed, by adding additional output to mallocmem, I can see that
> > it fails exactly when it attempts to malloc memory larger than
> > the CommitLimit value in /proc/meminfo.  I expect the same is
> > true for the dd test.
> >
> > Note that CommitLimit DOES go down when memory is ballooned-out
> > from a guest.  So your test does point out to me that I should
> > include a warning in the documentation not only that a swap disk
> > should be configured, but also that the swap disk should be
> > configured larger for a guest if selfballooning will be turned on.
> >
> > Thanks,
> > Dan
> >
> >> -----Original Message-----
> >> From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
> >> [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx]On Behalf Of
> >> viets@xxxxxxx
> >> Sent: Friday, May 16, 2008 3:36 AM
> >> To: xen-devel@xxxxxxxxxxxxxxxxxxx
> >> Subject: RE: [Xen-devel] [PATCH] balloon: selfballooning and
> >> post memory
> >> info via xenbus,
> >>
> >>
> >> Hello,
> >>
> >> thanks for the patch, I was waiting for this feature.
> >>
> >> I've tried this patch and I've seen that if I malloc a great
> >> size of memory in time, this fails, but if I malloc a small
> >> size first and then resize it slowly, it works.
> >>
> >> this highly suffisticated (:p) program I use to test the 
> ballooning:
> >>
> >> #include <stdio.h>
> >> #include <unistd.h>
> >>
> >> int main () {
> >>   void *v;
> >>   int i;
> >>   for(i=40; i < 50; ++i) {
> >>     v = malloc((i*32*1024*1024));
> >>     printf("%i\n", i);
> >>     if ( v != NULL) {
> >>       system("cat /proc/xen/balloon");
> >>       sleep(1);
> >>       free(v);
> >>     }
> >>   }
> >> }
> >>
> >> same effect I've got if I change the blocksize in a dd:
> >>
> >> works: dd if=zero of=/test.img count=1 bs=32M
> >> doesn't work: dd if=zero of=/test.img count=1 bs=256M
> >>
> >> Don't know whether this is the right test for this...
> >>
> >> greetings
> >> Torben Viets
> >>
> >>
> >> Dan Magenheimer wrote
> >>> OK, here's the promised patch.  The overall objective of the
> >>> patch is to enable limited memory load-balancing capabilities
> >>> as a step toward allowing limited memory overcommit.  With
> >>> this and some other minor hackery, I was able to run as
> >>> many as 15 lightly loaded 512MB domains on a 2GB system
> >>> (yes, veerrrryyy slooowwwlly).
> >>>
> >>> Review/comments appreciated.
> >>>
> >>> With this patch, balloon.c communicates (limited) useful
> >>> memory usage information via xenbus.  It also implements
> >>> "selfballooning" which applies the memory information
> >>> locally to immediately adjust the balloon, giving up memory
> >>> when it is not needed and asking for it back when it is needed,
> >>> implementing a first-come-first-served system-wide ballooning
> >>> "policy".  When a domain asks for memory but none is available,
> >>> it must use its own configured swap disk, resulting in
> >>> (potentially severe) performance degradation.  Naturally,
> >>> it is not recommended to turn on selfballooning in a domain
> >>> that has no swap disk or if performance is more important
> >>> than increasing the number of VMs runnable on a physical machine.
> >>>
> >>> A key assumption is that the Linux variable vm_committed_space
> >>> is a reasonable first approximation of memory needed by a domain.
> >>> This approximation will probably improve over time, but is
> >>> a good start for now.  The variable is bound on the lower end
> >>> by the recently submitted minimum_target() algorithm patch;
> >>> thus O-O-M conditions should not occur.
> >>>
> >>> The code is a bit complicated in a couple of places because of
> >>> race conditions involving xenstored startup relative to
> >>> turning on selfballooning locally.  Because the key variable
> >>> (vm_committed_space) is not exported by Linux, I implemented
> >>> a horrible hack which still allows the code to work in a
> >>> module, however I fully expect that this part of the patch
> >>> will not be accepted (which will limit the functionality to
> >>> pvm domains only... probably OK for now).
> >>>
> >>> Existing balloon functionality which is unchanged:
> >>> - Set target for VM from domain0
> >>> - Set target inside VM by writing to /proc/xen/balloon
> >>> Existing balloon info on xenbus which is unchanged:
> >>> - /local/domain/X/memory/target
> >>> To turn on selfballooning:
> >>> - Inside a VM: "echo 1 > /proc/xen/balloon"
> >>> - From domain0: "xenstore-write
> >> /local/domain/X/memory/selfballoon 1"
> >>> To turn off selfballooning:
> >>> - Inside a VM: "echo 0 > /proc/xen/balloon"
> >>> - From domain0: "xenstore-write
> >> /local/domain/X/memory/selfballoon 0"
> >>> New balloon info now on xenbus:
> >>> - /local/domain/X/memory/selfballoon [0 or 1]
> >>> - /local/domain/X/memory/actual [kB] *
> >>> - /local/domain/X/memory/minimum [kB] *
> >>> - /local/domain/X/memory/selftarget [kB] * (only valid if
> >>>   selfballoon==1)
> >>>  * writeable only by balloon driver in X when either
> >>>    selfballooning is first enabled, or target is changed
> >>>    by domain0
> >>>
> >>> Thanks,
> >>> Dan
> >>>
> >>> ===================================
> >>> Thanks... for the memory
> >>> I really could use more / My throughput's on the floor
> >>> The balloon is flat / My swap disk's fat / I've OOM's in store
> >>> Overcommitted so much
> >>> (with apologies to the late great Bob Hope)
> >> --
> >>
> >>
> >> _______________________________________________
> >> Xen-devel mailing list
> >> Xen-devel@xxxxxxxxxxxxxxxxxxx
> >> http://lists.xensource.com/xen-devel
> >>
> >
> >
> 
> 
>


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel