Re: [Xen-devel] [PATCH] balloon: selfballooning and post memory info via xenbus,

["viets@xxxxxxx" <viets@xxxxxxx>]

Hello,

are there any plans to bring this feature in the xen kernel?

Possible as an expermental marked feature?

I use it about 1 month without any trouble on live systems.

greetings
Viets

Viets wrote:
> hey,
> 
> thanks for the tip, I've already memory hotplug activated. Now it works
> fine with 7 domains, but no one uses more than 256MB... I'd like to test
> the ballooning with more than 2GB memory but at the moment I'ven't  a
> live machine which  needs so much memory...
> 
> but with maxmen and hotplug, this defines the maximum or?
> 
> greetings
> Torben Viets
> 
> Dan Magenheimer wrote:
>>>> memory = 256
>>>> maxmem = 8192
>>>>       
>>
>> By the way, I'm not sure if you knew this, but the above two
>> lines don't work as you might want.  The maxmem is ignored.
>> The domain is launched (in this example) with 256MB of
>> memory and (at least without hot-plug memory support in the
>> guest) memory can only be decreased from there, not increased.
>>
>> So to run a guest which adjusts between 256MB and 8192MB
>> of memory, you must launch it with 8192MB and balloon it
>> down to 256MB.  If Xen does not have 8192MB free at
>> launch, launching the domain will fail.
>>
>> Dan
>>
>>  
>>> -----Original Message-----
>>> From: Torben Viets [mailto:viets@xxxxxxx]
>>> Sent: Friday, May 16, 2008 10:51 AM
>>> To: xen-devel@xxxxxxxxxxxxxxxxxxx
>>> Cc: dan.magenheimer@xxxxxxxxxx
>>> Subject: Re: [Xen-devel] [PATCH] balloon: selfballooning and post memory
>>> info via xenbus,
>>>
>>>
>>> Dan Magenheimer wrote:
>>>    
>>>> (Your reply came to me but not to the list... not sure why.
>>>> So I've attached your full reply below.)
>>>>
>>>>       
>>> thanks, hope this time it works....
>>>    
>>>>> ah ok, that is my failure, I need a bigger swapdisk ;)
>>>>>
>>>>>         
>>>> Yes, definitely.  If you are creating the swapdisk on an ext3
>>>> filesystem, you might try using sparse files.  They won't
>>>> take up much disk space unless/until they get swapped-to.
>>>> There might be some performance ramifications though.
>>>> (My testing has been with swap disk as a logical volume
>>>> so I can't try sparse.)
>>>>
>>>>
>>>>      
>>>>> Ok, our plan is to have a high availbilty xen farm. Now we're
>>>>> beginning
>>>>> with 2 Suns X2200, each has 16GB RAM. The idea, why we like to use
>>>>> selfballooning, because of peak traffic on a server,         
>>> normal a server
>>>    
>>>>> needs about 256MB, but when it needs more, it shouldn't be a
>>>>> problem to
>>>>> give it 4GB. The idea is not to overbook the memory, but have the
>>>>> ability to get rid of memory failures because of peaks.
>>>>>
>>>>>         
>>>> Exactly what it is intended for!
>>>>
>>>> I'd be interested in how it works for guests with memory=4096
>>>> and higher.  All of my testing so far has been on a machine with
>>>> only 2GB of physical memory so I can test lots of guests but
>>>> no large guests.
>>>>
>>>>       
>>> I'll test it on monday, now I'm going into my weekend ;) but I think,
>>> that I wasn't able to get more than 2GB RAM allocated, but I will test
>>> it on monday again.
>>>
>>> PS: In my first mail I've attached my whole signatur, I remove it
>>> because I get enough spam ;)
>>>
>>> Thanks
>>> Torben Viets
>>>    
>>>> Thanks,
>>>> Dan
>>>>
>>>>
>>>>      
>>>>> -----Original Message-----
>>>>> From: viets@xxxxxxx [mailto:viets@xxxxxxx]
>>>>> Sent: Friday, May 16, 2008 9:49 AM
>>>>> To: dan.magenheimer@xxxxxxxxxx;         
>>> xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
>>>    
>>>>> Subject: Re: [Xen-devel] [PATCH] balloon: selfballooning and
>>>>> post memory
>>>>> info via xenbus,
>>>>>
>>>>>
>>>>> Dan Magenheimer wrote:
>>>>>
>>>>>        
>>>>>>> thanks for the patch, I was waiting for this feature.
>>>>>>>
>>>>>>>             
>>>>>> Thanks very much for the testing and feedback!  Could you
>>>>>> comment on what you plan to use it for?  (Keir hasn't accepted
>>>>>> it yet, so I am looking for user support ;-)
>>>>>>
>>>>>>           
>>>>> Ok, our plan is to have a high availbilty xen farm. Now we're
>>>>> beginning
>>>>> with 2 Suns X2200, each has 16GB RAM. The idea, why we like to use
>>>>> selfballooning, because of peak traffic on a server,         
>>> normal a server
>>>    
>>>>> needs about 256MB, but when it needs more, it shouldn't be a
>>>>> problem to
>>>>> give it 4GB. The idea is not to overbook the memory, but have the
>>>>> ability to get rid of memory failures because of peaks.
>>>>>
>>>>>
>>>>>        
>>>>>> First question: Do you have a swap (virtual) disk configured and,
>>>>>> if so, how big is it?  (Use "swapon -s" and the size shows in KB.)
>>>>>> Selfballooning shouldn't be run in a domain with no swap disk.
>>>>>> Also, how big is your "memory=" in your vm.cfg file?
>>>>>>
>>>>>>           
>>>>> #kernel = "/boot/xen-3.2.0/vmlinuz-2.6.18.8-xenU"
>>>>> #kernel = "/boot/vmlinuz-2.6.18.8-xenU"
>>>>> kernel = "/boot/vmlinuz-selfballooning"
>>>>> memory = 256
>>>>> maxmem = 8192
>>>>> vcpu = 4
>>>>> name = "test.work.de"
>>>>> vif = [ 'bridge=xenvlan323' ]
>>>>> disk = [ 'phy:/dev/sda,hda,w', 'file:/var/swap.img,hdb,w' ]
>>>>> root = "/dev/hda ro"
>>>>> extra = 'xencons=tty'
>>>>>
>>>>>
>>>>>
>>>>> swap_size = 256M
>>>>>
>>>>>
>>>>>        
>>>>>> I'm not able to reproduce your dd failure at all, even with
>>>>>> bs=2047M (dd doesn't permit larger values for bs).
>>>>>> Your program (I called it "mallocmem") does eventually fail for
>>>>>> me but not until i==88.  However, I have a 2GB swap disk           
>>> configured.
>>>    
>>>>> ah ok, that is my failure, I need a bigger swapdisk ;)
>>>>>
>>>>>        
>>>>>> I think both tests are really measuring the total virtual memory
>>>>>> space configured, e.g. the sum of physical memory (minus kernel
>>>>>> overhead) and configured swap space.  I think you will find that
>>>>>> both will fail similarly with ballooning off and even on           
>>> a physical
>>>    
>>>>>> system, just at different points in virtual memory usage.
>>>>>> Indeed, by adding additional output to mallocmem, I can see that
>>>>>> it fails exactly when it attempts to malloc memory larger than
>>>>>> the CommitLimit value in /proc/meminfo.  I expect the same is
>>>>>> true for the dd test.
>>>>>>
>>>>>> Note that CommitLimit DOES go down when memory is ballooned-out
>>>>>> from a guest.  So your test does point out to me that I should
>>>>>> include a warning in the documentation not only that a swap disk
>>>>>> should be configured, but also that the swap disk should be
>>>>>> configured larger for a guest if selfballooning will be turned on.
>>>>>>
>>>>>> Thanks,
>>>>>> Dan
>>>>>>
>>>>>>
>>>>>>          
>>>>>>> -----Original Message-----
>>>>>>> From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
>>>>>>> [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx]On Behalf Of
>>>>>>> viets@xxxxxxx
>>>>>>> Sent: Friday, May 16, 2008 3:36 AM
>>>>>>> To: xen-devel@xxxxxxxxxxxxxxxxxxx
>>>>>>> Subject: RE: [Xen-devel] [PATCH] balloon: selfballooning and
>>>>>>> post memory
>>>>>>> info via xenbus,
>>>>>>>
>>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> thanks for the patch, I was waiting for this feature.
>>>>>>>
>>>>>>> I've tried this patch and I've seen that if I malloc a great
>>>>>>> size of memory in time, this fails, but if I malloc a small
>>>>>>> size first and then resize it slowly, it works.
>>>>>>>
>>>>>>> this highly suffisticated (:p) program I use to test the
>>>>>>>
>>>>>>>             
>>>>> ballooning:
>>>>>
>>>>>        
>>>>>>> #include <stdio.h>
>>>>>>> #include <unistd.h>
>>>>>>>
>>>>>>> int main () {
>>>>>>>   void *v;
>>>>>>>   int i;
>>>>>>>   for(i=40; i < 50; ++i) {
>>>>>>>     v = malloc((i*32*1024*1024));
>>>>>>>     printf("%i\n", i);
>>>>>>>     if ( v != NULL) {
>>>>>>>       system("cat /proc/xen/balloon");
>>>>>>>       sleep(1);
>>>>>>>       free(v);
>>>>>>>     }
>>>>>>>   }
>>>>>>> }
>>>>>>>
>>>>>>> same effect I've got if I change the blocksize in a dd:
>>>>>>>
>>>>>>> works: dd if=zero of=/test.img count=1 bs=32M
>>>>>>> doesn't work: dd if=zero of=/test.img count=1 bs=256M
>>>>>>>
>>>>>>> Don't know whether this is the right test for this...
>>>>>>>
>>>>>>> greetings
>>>>>>> Torben Viets
>>>>>>>
>>>>>>>
>>>>>>> Dan Magenheimer wrote
>>>>>>>
>>>>>>>            
>>>>>>>> OK, here's the promised patch.  The overall objective of the
>>>>>>>> patch is to enable limited memory load-balancing capabilities
>>>>>>>> as a step toward allowing limited memory overcommit.  With
>>>>>>>> this and some other minor hackery, I was able to run as
>>>>>>>> many as 15 lightly loaded 512MB domains on a 2GB system
>>>>>>>> (yes, veerrrryyy slooowwwlly).
>>>>>>>>
>>>>>>>> Review/comments appreciated.
>>>>>>>>
>>>>>>>> With this patch, balloon.c communicates (limited) useful
>>>>>>>> memory usage information via xenbus.  It also implements
>>>>>>>> "selfballooning" which applies the memory information
>>>>>>>> locally to immediately adjust the balloon, giving up memory
>>>>>>>> when it is not needed and asking for it back when it is needed,
>>>>>>>> implementing a first-come-first-served system-wide ballooning
>>>>>>>> "policy".  When a domain asks for memory but none is available,
>>>>>>>> it must use its own configured swap disk, resulting in
>>>>>>>> (potentially severe) performance degradation.  Naturally,
>>>>>>>> it is not recommended to turn on selfballooning in a domain
>>>>>>>> that has no swap disk or if performance is more important
>>>>>>>> than increasing the number of VMs runnable on a               
>>> physical machine.
>>>    
>>>>>>>> A key assumption is that the Linux variable vm_committed_space
>>>>>>>> is a reasonable first approximation of memory needed by
>>>>>>>>               
>>> a domain.
>>>    
>>>>>>>> This approximation will probably improve over time, but is
>>>>>>>> a good start for now.  The variable is bound on the lower end
>>>>>>>> by the recently submitted minimum_target() algorithm patch;
>>>>>>>> thus O-O-M conditions should not occur.
>>>>>>>>
>>>>>>>> The code is a bit complicated in a couple of places because of
>>>>>>>> race conditions involving xenstored startup relative to
>>>>>>>> turning on selfballooning locally.  Because the key variable
>>>>>>>> (vm_committed_space) is not exported by Linux, I implemented
>>>>>>>> a horrible hack which still allows the code to work in a
>>>>>>>> module, however I fully expect that this part of the patch
>>>>>>>> will not be accepted (which will limit the functionality to
>>>>>>>> pvm domains only... probably OK for now).
>>>>>>>>
>>>>>>>> Existing balloon functionality which is unchanged:
>>>>>>>> - Set target for VM from domain0
>>>>>>>> - Set target inside VM by writing to /proc/xen/balloon
>>>>>>>> Existing balloon info on xenbus which is unchanged:
>>>>>>>> - /local/domain/X/memory/target
>>>>>>>> To turn on selfballooning:
>>>>>>>> - Inside a VM: "echo 1 > /proc/xen/balloon"
>>>>>>>> - From domain0: "xenstore-write
>>>>>>>>
>>>>>>>>               
>>>>>>> /local/domain/X/memory/selfballoon 1"
>>>>>>>
>>>>>>>            
>>>>>>>> To turn off selfballooning:
>>>>>>>> - Inside a VM: "echo 0 > /proc/xen/balloon"
>>>>>>>> - From domain0: "xenstore-write
>>>>>>>>
>>>>>>>>               
>>>>>>> /local/domain/X/memory/selfballoon 0"
>>>>>>>
>>>>>>>            
>>>>>>>> New balloon info now on xenbus:
>>>>>>>> - /local/domain/X/memory/selfballoon [0 or 1]
>>>>>>>> - /local/domain/X/memory/actual [kB] *
>>>>>>>> - /local/domain/X/memory/minimum [kB] *
>>>>>>>> - /local/domain/X/memory/selftarget [kB] * (only valid if
>>>>>>>>   selfballoon==1)
>>>>>>>>  * writeable only by balloon driver in X when either
>>>>>>>>    selfballooning is first enabled, or target is changed
>>>>>>>>    by domain0
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Dan
>>>>>>>>
>>>>>>>> ===================================
>>>>>>>> Thanks... for the memory
>>>>>>>> I really could use more / My throughput's on the floor
>>>>>>>> The balloon is flat / My swap disk's fat / I've OOM's in store
>>>>>>>> Overcommitted so much
>>>>>>>> (with apologies to the late great Bob Hope)
>>>>>>>>
>>>>>>>>               
>>>>>>> -- 
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Xen-devel mailing list
>>>>>>> Xen-devel@xxxxxxxxxxxxxxxxxxx
>>>>>>> http://lists.xensource.com/xen-devel
>>>>>>>
>>>>>>>
>>>>>>>             
>>>>>         
>>>> _______________________________________________
>>>> Xen-devel mailing list
>>>> Xen-devel@xxxxxxxxxxxxxxxxxxx
>>>> http://lists.xensource.com/xen-devel
>>>>
>>>>
>>>>       
>>>     
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@xxxxxxxxxxxxxxxxxxx
>> http://lists.xensource.com/xen-devel
>>
>>   
> 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel