[Xen-users] xen-4.0.1 from make world - hypervisor alone consumes around 800 MB RAM , OOMs when try to baloon to lower amount, no domUs running yet

[Petr Vacek <tachyon.eagle@xxxxxxxxx>]

Greetings,
I've just setuped a new AMD opteron 6168 12-core server with Xen-4.0.1 - downloading sources , running 'make world' - adding raid+ata modules into kernel and running ...
Everything works fine (filesystem,raid,domUs crunching numbers) , save for one troubling memory issue with dom0 :

When running with hypervisor enabled , I see used memory as 860 MB (in top) right after linux start, even without Xend service and any domU running,
when I strech domain 0 under 800 MB , I get OOM killer and restart.  This way I know that I cannot possibly use this memory and it's eaten up - 

I cannot give it to domU certainly .

I can see this amount in meminfo in directMap4k , but it does not add up with other parts of meminfo to me - it does not belong to any visible process.
When I use same booting configuration without Xen hypervisor, this memory usage goes away - linux eats about 60 MB , only service running is SSH and very basic Ubuntu 10.10. 

Kernel is from xen sources : 2.6.32.27 - kernel config as here ( http://pavouk.mediawork.cz/xen/2011/kernel-config.txt )
I've placed dmesg, xm dmesg, dmidecode machine information here: http://pavouk.mediawork.cz/xen/2011/   

I've tried building 4.0.2 - memory issue was still present + kernel bugs so I reverted to older 4.0.1 runnning otherwise stable for my setup.

Could this be caused by some kernel misconfiguration or xen hypervisor misconfiguration ? 

I've noticed some warning during initialization , but I think it's something different :
Performance Events: AMD PMU driver.                                                                                                                              

[    0.006998] ------------[ cut here ]------------                                                                                                                             
[    0.006998] WARNING: at /opt/xen/xen-4.0.1/linux-2.6-pvops.git/arch/x86/xen/enlighten.c:729 xen_apic_write+0x15/0x17()                                                       

[    0.006998] Hardware name: H8SGL                                                                                                                                             
[    0.006998] Modules linked in:                                                                                                                                               

[    0.006998] Pid: 0, comm: swapper Not tainted 2.6.32.27 #9                                                                                                                   
[    0.006998] Call Trace:                                                                                                                                                      

[    0.006998]  [<ffffffff8107970b>] warn_slowpath_common+0x77/0x8f                                                                                                             
[    0.006998]  [<ffffffff81079732>] warn_slowpath_null+0xf/0x11                                                                                                                

[    0.006998]  [<ffffffff810355c2>] xen_apic_write+0x15/0x17                                                                                                                   
[    0.006998]  [<ffffffff8104872a>] perf_events_lapic_init+0x2e/0x30                                                                                                           

[    0.006998]  [<ffffffff81a976a2>] init_hw_perf_events+0x33e/0x3db                                                                                                            
[    0.006998]  [<ffffffff81a971c0>] identify_boot_cpu+0x3c/0x3e                                                                                                                

[    0.006998]  [<ffffffff81a97340>] check_bugs+0x9/0x2d                                                                                                                        
[    0.006998]  [<ffffffff81a8ddbf>] start_kernel+0x432/0x447                                                                                                                   

[    0.006998]  [<ffffffff81a8d2c1>] x86_64_start_reservations+0xac/0xb0                                                                                                        
[    0.006998]  [<ffffffff81a91ad0>] xen_start_kernel+0x5b7/0x5be                                                                                                               

[    0.006998] ---[ end trace 4eaa2a86a8e2da22 ]---


The troubling meminfo dump:

cat /proc/meminfo :
MemTotal:        1229376 kB
MemFree:          345440 kB
Buffers:             412 kB
Cached:             8572 kB

SwapCached:            0 kB
Active:            45304 kB
Inactive:          14112 kB
Active(anon):      38464 kB
Inactive(anon):    12304 kB
Active(file):       6840 kB
Inactive(file):     1808 kB
Unevictable:          20 kB

Mlocked:              20 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:               948 kB
Writeback:             0 kB
AnonPages:         50508 kB
Mapped:             6600 kB
Shmem:               320 kB

Slab:              15904 kB
SReclaimable:       5236 kB
SUnreclaim:        10668 kB
KernelStack:        1016 kB
PageTables:         1664 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB

CommitLimit:      614688 kB
Committed_AS:     153440 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      264292 kB
VmallocChunk:   34359473836 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0

HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:     8647280 kB
DirectMap2M:           0 kB

Thanks in advance for any hints how to tamper this memory burden
Regards
Petr Vacek
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users