[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-devel] Memory allocation with rebooted HVM



Is there a known bug in xen-unstable with rebooting of HVMs w/stubdom, and having the new domU come up before the old domU has been shut down and its memory fully reclaimed? I have had a couple of problems recently when customers rebooted their HVM domU through the OS; the last time, Xend froze, complaining in the log that it couldn't free up enough memory to start the rebooted domain. Of course, that memory should have been free, if the old domain had been fully shut down.
 
The xend.log looks like what I'd expect with an out of memory situation:
 
[2009-10-11 14:23:19 6419] DEBUG (XendDomainInfo:121) XendDomainInfo.create_from_dict(...
[2009-10-11 14:23:19 6419] DEBUG (XendDomainInfo:2369) XendDomainInfo.constructDomain
[2009-10-11 14:23:19 6419] DEBUG (balloon:181) Balloon: 602420 KiB free; need 4096; done.
[2009-10-11 14:23:19 6419] DEBUG (XendDomain:454) Adding Domain: 21
[2009-10-11 14:23:19 6419] DEBUG (XendDomainInfo:2614) XendDomainInfo.initDomain: 21 256
[2009-10-11 14:23:19 6419] DEBUG (image:336) Stored a VNC password for vfb access
[2009-10-11 14:23:19 6419] DEBUG (image:843) args: boot, val: cd
[2009-10-11 14:23:19 6419] DEBUG (image:843) args: fda, val: None
[2009-10-11 14:23:19 6419] DEBUG (image:843) args: fdb, val: None
[2009-10-11 14:23:19 6419] DEBUG (image:843) args: soundhw, val: None
[2009-10-11 14:23:19 6419] DEBUG (image:843) args: localtime, val: 0
[2009-10-11 14:23:19 6419] DEBUG (image:843) args: serial, val: None
[2009-10-11 14:23:19 6419] DEBUG (image:843) args: std-vga, val: 0
[2009-10-11 14:23:19 6419] DEBUG (image:843) args: isa, val: 0
[2009-10-11 14:23:19 6419] DEBUG (image:843) args: acpi, val: 1
[2009-10-11 14:23:19 6419] DEBUG (image:843) args: usb, val: 0
[2009-10-11 14:23:19 6419] DEBUG (image:843) args: usbdevice, val: tablet
[2009-10-11 14:23:19 6419] DEBUG (image:843) args: gfx_passthru, val: None
[2009-10-11 14:23:19 6419] INFO (image:779) Need to create platform device.[domid:21]
[2009-10-11 14:23:19 6419] DEBUG (XendDomainInfo:2641) _initDomain:shadow_memory=0x9, memory_static_max=0x40000000, memory_static_min=0x0.
 [2009-10-11 14:23:19 6419] DEBUG (balloon:135) Balloon: tmem relinquished -1 KiB of 464976 KiB requested.
[2009-10-11 14:23:19 6419] DEBUG (balloon:187) Balloon: 601008 KiB free; 0 to scrub; need 1065984; retries: 20.
[2009-10-11 14:23:19 6419] DEBUG (balloon:187) Balloon: 601008 KiB free; 0 to scrub; need 1065984; retries: 20.
[2009-10-11 14:23:42 6419] ERROR (XendDomainInfo:2694) XendDomainInfo.initDomain: exception occurred
Traceback (most recent call last):
  File "/usr/lib64/python2.6/site-packages/xen/xend/XendDomainInfo.py", line 2655, in _initDomain
    balloon.free(memory + shadow + vtd_mem, self)
  File "/usr/lib64/python2.6/site-packages/xen/xend/balloon.py", line 225, in free
    free_mem + scrub_mem + dom0_alloc - dom0_min_mem))
VmError: I need 1065984 KiB, but dom0_min_mem is 196608 and shrinking to 196608 KiB would leave only 924848 KiB free.
[2009-10-11 14:23:42 6419] ERROR (XendDomainInfo:467) VM start failed
Traceback (most recent call last):
  File "/usr/lib64/python2.6/site-packages/xen/xend/XendDomainInfo.py", line 453, in start
    XendTask.log_progress(31, 60, self._initDomain)
  File "/usr/lib64/python2.6/site-packages/xen/xend/XendTask.py", line 209, in log_progress
    retval = func(*args, **kwds)
  File "/usr/lib64/python2.6/site-packages/xen/xend/XendDomainInfo.py", line 2697, in _initDomain
    raise exn
VmError: I need 1065984 KiB, but dom0_min_mem is 196608 and shrinking to 196608 KiB would leave only 924848 KiB free.
[2009-10-11 14:23:42 6419] DEBUG (XendDomainInfo:2844) XendDomainInfo.destroy: domid=21
[2009-10-11 14:23:42 6419] DEBUG (XendDomainInfo:2279) Destroying device model
[2009-10-11 14:23:42 6419] ERROR (XendDomainInfo:2282) Device model destroy failed X86_HVM_ImageHandler instance has no attribute 'sentinel_lock'
Traceback (most recent call last):
  File "/usr/lib64/python2.6/site-packages/xen/xend/XendDomainInfo.py", line 2280, in _releaseDevices
    self.image.destroyDeviceModel()
[2009-10-11 14:23:42 6419] DEBUG (XendDomainInfo:2286) Releasing devices
[2009-10-11 14:23:42 6419] ERROR (XendDomainInfo:126) Domain construction failed
Traceback (most recent call last):
  File "/usr/lib64/python2.6/site-packages/xen/xend/XendDomainInfo.py", line 124, in create_from_dict
    vm.start()
  File "/usr/lib64/python2.6/site-packages/xen/xend/XendDomainInfo.py", line 453, in start
    XendTask.log_progress(31, 60, self._initDomain)
  File "/usr/lib64/python2.6/site-packages/xen/xend/XendTask.py", line 209, in log_progress
    retval = func(*args, **kwds)
  File "/usr/lib64/python2.6/site-packages/xen/xend/XendDomainInfo.py", line 2697, in _initDomain
    raise exn
VmError: I need 1065984 KiB, but dom0_min_mem is 196608 and shrinking to 196608 KiB would leave only 924848 KiB free.
[2009-10-11 14:23:42 6419] DEBUG (XendDomainInfo:2844) XendDomainInfo.destroy: domid=20
[2009-10-11 14:23:42 6419] ERROR (XendDomainInfo:2117) Failed to restart domain 19.
Traceback (most recent call last):
  File "/usr/lib64/python2.6/site-packages/xen/xend/XendDomainInfo.py", line 2100, in _restart
    new_dom_info)
  File "/usr/lib64/python2.6/site-packages/xen/xend/XendDomain.py", line 989, in domain_create_from_dict
    dominfo = XendDomainInfo.create_from_dict(config_dict)
  File "/usr/lib64/python2.6/site-packages/xen/xend/XendDomainInfo.py", line 124, in create_from_dict
    vm.start()
  File "/usr/lib64/python2.6/site-packages/xen/xend/XendDomainInfo.py", line 453, in start
    XendTask.log_progress(31, 60, self._initDomain)
  File "/usr/lib64/python2.6/site-packages/xen/xend/XendTask.py", line 209, in log_progress
    retval = func(*args, **kwds)
  File "/usr/lib64/python2.6/site-packages/xen/xend/XendDomainInfo.py", line 2697, in _initDomain
    raise exn
VmError: I need 1065984 KiB, but dom0_min_mem is 196608 and shrinking to 196608 KiB would leave only 924848 KiB free.
After that, Xend was unresponsive. Killing it and restarting it killed all the other domU's, but new ones still couldn't be created, forcing me to reboot the physical machine.
 
Unfortunately, this doesn't seem to happen every time. I'm still trying to determine how I can reproduce it reliably.
 
-John
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.