|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-users] unexpected Out Of Memory (OOM)
Le mercredi 07 aoÃt 2013 Ã 16:26 +0200, Olivier Bonvalet a Ãcrit : > > Le mercredi 07 aoÃt 2013 Ã 14:36 +0100, Ian Campbell a Ãcrit : > > On Wed, 2013-08-07 at 13:17 +0200, Olivier Bonvalet wrote: > > > > > name = 'reto' > > > vcpus = 1 > > > maxvcpus = 8 > > > memory = 8192 > > > vif = [ 'mac=0e:00:00:00:8e:70,bridge=vlan' ] > > > disk = [ > > > '/dev/rbd/sas3copies/reto-root,,xvda,w', > > > '/dev/rbd/sas3copies/reto-home,,xvdb,w', > > > '/dev/rbd/sas3copies/reto-var,,xvdc,w', > > > '/dev/rbd/sas3copies/reto-mysql,,xvdd,w', > > > '/dev/rbd/sas3copies/reto-exim,,xvde,w', > > > > > > '/dev/loop1,raw,xvdy,r', > > > '/dev/loop2,raw,xvdz,r' ] > > > kernel = '/etc/xen/kernels/reto/vmlinuz' > > > ramdisk = '/etc/xen/kernels/reto/initrd.img' > > > root = '/dev/xvda ro rootfstype=ext4' > > > extra = 'panic=60' > > > > All looks pretty normal. > > > > > Then the console : > > > > > > Parsing config from /etc/xen/reto.cfg > > > Daemon running with PID 20283 > > > [ 0.000000] Initializing cgroup subsys cpu > > > [ 0.000000] Linux version 2.6.50-dae-xen (root@yiu) (gcc version 4.8.1 > > > (Debian 4.8.1-8) ) #2 SMP Sun Aug 4 22:42:05 CEST 2013 > > > [ 0.000000] Command line: root=/dev/xvda ro rootfstype=ext4 panic=60 > > > [ 0.000000] KERNEL supported cpus: > > > [ 0.000000] Intel GenuineIntel > > > [ 0.000000] ACPI in unprivileged domain disabled > > > [ 0.000000] e820: BIOS-provided physical RAM map: > > > [ 0.000000] Xen: [mem 0x0000000000000000-0x000000000009ffff] usable > > > [ 0.000000] Xen: [mem 0x00000000000a0000-0x00000000000fffff] reserved > > > [ 0.000000] Xen: [mem 0x0000000000100000-0x00000001ffffffff] usable > > > > This all looks good. > > > > > [ 1.776382] blkfront: xvdb: barrier or flush: disabled using > > > persistent grants > > > [ 1.797557] xvdb: unknown partition table > > > [ 1.798526] blkfront: xvdc: barrier or flush: disabled using > > > persistent grants > > > [ 1.820815] xvdc: unknown partition table > > > [ 1.822259] blkfront: xvdd: barrier or flush: disabled using > > > persistent grants > > > [ 1.853766] xvdd: unknown partition table > > > [ 1.854749] blkfront: xvde: barrier or flush: disabled using > > > persistent grants > > > [ 1.857572] xvde: unknown partition table > > > [ 1.858010] Setting capacity to 2097152 > > > [ 1.858018] xvda: detected capacity change from 0 to 1073741824 > > > [ 1.858760] blkfront: xvdy: flush diskcache: enabled using persistent > > > grants > > > [ 1.860885] xvdy: unknown partition table > > > [ 1.861913] blkfront: xvdz: flush diskcache: enabled using persistent > > > grants > > > [ 1.863770] xvdz: unknown partition table > > > > I know persistent grants have a fixed memory overhead, not sure if that > > is on the back or frontend though. Roger? > > > > is there any way to disable persistent grants manually for testing > > purposes? > > > > > [ 8.595343] zram: module is from the staging directory, the quality is > > > unknown, you have been warned. > > > [ 8.596003] zram: Created 1 device(s) ... > > > > My first thought was that there was a memory leak in the kernel > > somewhere, but staging drivers doing "magic" things with memory make me > > nervous. Can you try without zram to rule it out please? > > > > Ian. > > > > > > You're right, I disabled zram on this Domu (others didn't have zram). > The problem is still present. > > > > _______________________________________________ > Xen-users mailing list > Xen-users@xxxxxxxxxxxxx > http://lists.xen.org/xen-users So I recompiled a kernel with the kmemleak feature. I obtain that kind of list, but not sure that it's usefull : unreferenced object 0xffff88007cca8a80 (size 8): comm "swapper/0", pid 1, jiffies 4294892300 (age 150.688s) hex dump (first 8 bytes): 74 69 6d 65 72 31 00 a5 timer1.. backtrace: [<ffffffff811b70d6>] kasprintf+0x3e/0x40 [<ffffffff810e296d>] __kmalloc_track_caller+0xe7/0xf3 [<ffffffff811b7071>] kvasprintf+0x41/0x68 [<ffffffff811b70d6>] kasprintf+0x3e/0x40 [<ffffffff810056eb>] xen_setup_timer+0x62/0xc3 [<ffffffff8134296e>] xen_cpu_up+0x5b/0x367 [<ffffffff81348662>] _cpu_up+0xb8/0x124 [<ffffffff81348792>] cpu_up+0xc4/0xd0 [<ffffffff8169dca6>] smp_init+0x4a/0x75 [<ffffffff81689d55>] kernel_init_freeable+0x78/0x18d [<ffffffff8133d92a>] kernel_init+0x0/0xd1 [<ffffffff8133d92f>] kernel_init+0x5/0xd1 [<ffffffff81354efc>] ret_from_fork+0x7c/0xb0 [<ffffffff8133d92a>] kernel_init+0x0/0xd1 [<ffffffffffffffff>] 0xffffffffffffffff unreferenced object 0xffff88007c527d90 (size 16): comm "swapper/0", pid 1, jiffies 4294892302 (age 150.680s) hex dump (first 16 bytes): 72 65 73 63 68 65 64 31 00 6b 6b 6b 6b 6b 6b a5 resched1.kkkkkk. backtrace: [<ffffffff811b70d6>] kasprintf+0x3e/0x40 [<ffffffff810e296d>] __kmalloc_track_caller+0xe7/0xf3 [<ffffffff811b7071>] kvasprintf+0x41/0x68 [<ffffffff811b70d6>] kasprintf+0x3e/0x40 [<ffffffff8100d03e>] text_poke+0x186/0x195 [<ffffffff8134d595>] xen_smp_intr_init+0x39/0x271 [<ffffffff81342c28>] xen_cpu_up+0x315/0x367 [<ffffffff81348662>] _cpu_up+0xb8/0x124 [<ffffffff81348792>] cpu_up+0xc4/0xd0 [<ffffffff8169dca6>] smp_init+0x4a/0x75 [<ffffffff81689d55>] kernel_init_freeable+0x78/0x18d [<ffffffff8133d92a>] kernel_init+0x0/0xd1 [<ffffffff8133d92f>] kernel_init+0x5/0xd1 [<ffffffff81354efc>] ret_from_fork+0x7c/0xb0 [<ffffffff8133d92a>] kernel_init+0x0/0xd1 [<ffffffffffffffff>] 0xffffffffffffffff unreferenced object 0xffff88007c526158 (size 16): comm "swapper/0", pid 1, jiffies 4294892303 (age 150.676s) hex dump (first 16 bytes): 63 61 6c 6c 66 75 6e 63 31 00 6b 6b 6b 6b 6b a5 callfunc1.kkkkk. backtrace: [<ffffffff811b70d6>] kasprintf+0x3e/0x40 [<ffffffff810e296d>] __kmalloc_track_caller+0xe7/0xf3 [<ffffffff811b7071>] kvasprintf+0x41/0x68 [<ffffffff811b70d6>] kasprintf+0x3e/0x40 [<ffffffff8134d5db>] xen_smp_intr_init+0x7f/0x271 [<ffffffff81342c28>] xen_cpu_up+0x315/0x367 [<ffffffff81348662>] _cpu_up+0xb8/0x124 [<ffffffff81348792>] cpu_up+0xc4/0xd0 [<ffffffff8169dca6>] smp_init+0x4a/0x75 [<ffffffff81689d55>] kernel_init_freeable+0x78/0x18d [<ffffffff8133d92a>] kernel_init+0x0/0xd1 [<ffffffff8133d92f>] kernel_init+0x5/0xd1 [<ffffffff81354efc>] ret_from_fork+0x7c/0xb0 [<ffffffff8133d92a>] kernel_init+0x0/0xd1 [<ffffffffffffffff>] 0xffffffffffffffff unreferenced object 0xffff88007cca8bd0 (size 8): comm "swapper/0", pid 1, jiffies 4294892303 (age 150.676s) hex dump (first 8 bytes): 64 65 62 75 67 31 00 a5 debug1.. backtrace: [<ffffffff811b70d6>] kasprintf+0x3e/0x40 [<ffffffff810e296d>] __kmalloc_track_caller+0xe7/0xf3 [<ffffffff811b7071>] kvasprintf+0x41/0x68 [<ffffffff811b70d6>] kasprintf+0x3e/0x40 [<ffffffff8134d624>] xen_smp_intr_init+0xc8/0x271 [<ffffffff81342c28>] xen_cpu_up+0x315/0x367 [<ffffffff81348662>] _cpu_up+0xb8/0x124 [<ffffffff81348792>] cpu_up+0xc4/0xd0 [<ffffffff8169dca6>] smp_init+0x4a/0x75 [<ffffffff81689d55>] kernel_init_freeable+0x78/0x18d [<ffffffff8133d92a>] kernel_init+0x0/0xd1 [<ffffffff8133d92f>] kernel_init+0x5/0xd1 [<ffffffff81354efc>] ret_from_fork+0x7c/0xb0 [<ffffffff8133d92a>] kernel_init+0x0/0xd1 [<ffffffffffffffff>] 0xffffffffffffffff unreferenced object 0xffff88007c527c38 (size 16): comm "swapper/0", pid 1, jiffies 4294892303 (age 150.680s) hex dump (first 16 bytes): 63 61 6c 6c 66 75 6e 63 73 69 6e 67 6c 65 31 00 callfuncsingle1. backtrace: [<ffffffff811b70d6>] kasprintf+0x3e/0x40 [<ffffffff810e296d>] __kmalloc_track_caller+0xe7/0xf3 [<ffffffff811b7071>] kvasprintf+0x41/0x68 [<ffffffff811b70d6>] kasprintf+0x3e/0x40 [<ffffffff812126ef>] bind_virq_to_irqhandler+0x3c/0x58 [<ffffffff8134d66d>] xen_smp_intr_init+0x111/0x271 [<ffffffff81342c28>] xen_cpu_up+0x315/0x367 [<ffffffff81348662>] _cpu_up+0xb8/0x124 [<ffffffff81348792>] cpu_up+0xc4/0xd0 [<ffffffff8169dca6>] smp_init+0x4a/0x75 [<ffffffff81689d55>] kernel_init_freeable+0x78/0x18d [<ffffffff8133d92a>] kernel_init+0x0/0xd1 [<ffffffff8133d92f>] kernel_init+0x5/0xd1 [<ffffffff81354efc>] ret_from_fork+0x7c/0xb0 [<ffffffff8133d92a>] kernel_init+0x0/0xd1 [<ffffffffffffffff>] 0xffffffffffffffff unreferenced object 0xffff88007c5262b0 (size 16): comm "swapper/0", pid 1, jiffies 4294892303 (age 150.680s) hex dump (first 16 bytes): 69 72 71 77 6f 72 6b 31 00 6b 6b 6b 6b 6b 6b a5 irqwork1.kkkkkk. backtrace: [<ffffffff811b70d6>] kasprintf+0x3e/0x40 [<ffffffff810e296d>] __kmalloc_track_caller+0xe7/0xf3 [<ffffffff811b7071>] kvasprintf+0x41/0x68 [<ffffffff811b70d6>] kasprintf+0x3e/0x40 [<ffffffff8134d6c6>] xen_smp_intr_init+0x16a/0x271 [<ffffffff81342c28>] xen_cpu_up+0x315/0x367 [<ffffffff81348662>] _cpu_up+0xb8/0x124 [<ffffffff81348792>] cpu_up+0xc4/0xd0 [<ffffffff8169dca6>] smp_init+0x4a/0x75 [<ffffffff81689d55>] kernel_init_freeable+0x78/0x18d [<ffffffff8133d92a>] kernel_init+0x0/0xd1 [<ffffffff8133d92f>] kernel_init+0x5/0xd1 [<ffffffff81354efc>] ret_from_fork+0x7c/0xb0 [<ffffffff8133d92a>] kernel_init+0x0/0xd1 [<ffffffffffffffff>] 0xffffffffffffffff unreferenced object 0xffff88007cca93b0 (size 8): comm "swapper/0", pid 1, jiffies 4294892303 (age 150.680s) hex dump (first 8 bytes): 74 69 6d 65 72 32 00 a5 timer2.. backtrace: [<ffffffff811b70d6>] kasprintf+0x3e/0x40 [<ffffffff810e296d>] __kmalloc_track_caller+0xe7/0xf3 [<ffffffff811b7071>] kvasprintf+0x41/0x68 [<ffffffff811b70d6>] kasprintf+0x3e/0x40 [<ffffffff810056eb>] xen_setup_timer+0x62/0xc3 [<ffffffff8134296e>] xen_cpu_up+0x5b/0x367 [<ffffffff81348662>] _cpu_up+0xb8/0x124 [<ffffffff81348792>] cpu_up+0xc4/0xd0 [<ffffffff8169dca6>] smp_init+0x4a/0x75 [<ffffffff81689d55>] kernel_init_freeable+0x78/0x18d [<ffffffff8133d92a>] kernel_init+0x0/0xd1 [<ffffffff8133d92f>] kernel_init+0x5/0xd1 [<ffffffff81354efc>] ret_from_fork+0x7c/0xb0 [<ffffffff8133d92a>] kernel_init+0x0/0xd1 [<ffffffffffffffff>] 0xffffffffffffffff unreferenced object 0xffff88007c526560 (size 16): comm "swapper/0", pid 1, jiffies 4294892303 (age 150.680s) hex dump (first 16 bytes): 73 70 69 6e 6c 6f 63 6b 32 00 6b 6b 6b 6b 6b a5 spinlock2.kkkkk. backtrace: [<ffffffff811b70d6>] kasprintf+0x3e/0x40 [<ffffffff810e296d>] __kmalloc_track_caller+0xe7/0xf3 [<ffffffff811b7071>] kvasprintf+0x41/0x68 [<ffffffff811b70d6>] kasprintf+0x3e/0x40 [<ffffffff81342cde>] xen_init_lock_cpu+0x64/0xbc [<ffffffff81342975>] xen_cpu_up+0x62/0x367 [<ffffffff81348662>] _cpu_up+0xb8/0x124 [<ffffffff81348792>] cpu_up+0xc4/0xd0 [<ffffffff8169dca6>] smp_init+0x4a/0x75 [<ffffffff81689d55>] kernel_init_freeable+0x78/0x18d [<ffffffff8133d92a>] kernel_init+0x0/0xd1 [<ffffffff8133d92f>] kernel_init+0x5/0xd1 [<ffffffff81354efc>] ret_from_fork+0x7c/0xb0 [<ffffffff8133d92a>] kernel_init+0x0/0xd1 [<ffffffffffffffff>] 0xffffffffffffffff unreferenced object 0xffff88007c527830 (size 16): comm "swapper/0", pid 1, jiffies 4294892303 (age 150.680s) hex dump (first 16 bytes): 72 65 73 63 68 65 64 32 00 6b 6b 6b 6b 6b 6b a5 resched2.kkkkkk. backtrace: [<ffffffff811b70d6>] kasprintf+0x3e/0x40 [<ffffffff810e296d>] __kmalloc_track_caller+0xe7/0xf3 [<ffffffff811b7071>] kvasprintf+0x41/0x68 [<ffffffff811b70d6>] kasprintf+0x3e/0x40 [<ffffffff8134d595>] xen_smp_intr_init+0x39/0x271 [<ffffffff81342c28>] xen_cpu_up+0x315/0x367 [<ffffffff81348662>] _cpu_up+0xb8/0x124 [<ffffffff81348792>] cpu_up+0xc4/0xd0 [<ffffffff8169dca6>] smp_init+0x4a/0x75 [<ffffffff81689d55>] kernel_init_freeable+0x78/0x18d [<ffffffff8133d92a>] kernel_init+0x0/0xd1 [<ffffffff8133d92f>] kernel_init+0x5/0xd1 [<ffffffff81354efc>] ret_from_fork+0x7c/0xb0 [<ffffffff8133d92a>] kernel_init+0x0/0xd1 [<ffffffffffffffff>] 0xffffffffffffffff unreferenced object 0xffff88007c5266b8 (size 16): comm "swapper/0", pid 1, jiffies 4294892303 (age 150.680s) hex dump (first 16 bytes): 63 61 6c 6c 66 75 6e 63 32 00 6b 6b 6b 6b 6b a5 callfunc2.kkkkk. backtrace: [<ffffffff811b70d6>] kasprintf+0x3e/0x40 [<ffffffff810e296d>] __kmalloc_track_caller+0xe7/0xf3 [<ffffffff811b7071>] kvasprintf+0x41/0x68 [<ffffffff811b70d6>] kasprintf+0x3e/0x40 [<ffffffff8134d5db>] xen_smp_intr_init+0x7f/0x271 [<ffffffff81342c28>] xen_cpu_up+0x315/0x367 [<ffffffff81348662>] _cpu_up+0xb8/0x124 [<ffffffff81348792>] cpu_up+0xc4/0xd0 [<ffffffff8169dca6>] smp_init+0x4a/0x75 [<ffffffff81689d55>] kernel_init_freeable+0x78/0x18d [<ffffffff8133d92a>] kernel_init+0x0/0xd1 [<ffffffff8133d92f>] kernel_init+0x5/0xd1 [<ffffffff81354efc>] ret_from_fork+0x7c/0xb0 [<ffffffff8133d92a>] kernel_init+0x0/0xd1 [<ffffffffffffffff>] 0xffffffffffffffff unreferenced object 0xffff88007cca8d20 (size 8): comm "swapper/0", pid 1, jiffies 4294892303 (age 150.680s) hex dump (first 8 bytes): 64 65 62 75 67 32 00 a5 debug2.. backtrace: [<ffffffff811b70d6>] kasprintf+0x3e/0x40 [<ffffffff810e296d>] __kmalloc_track_caller+0xe7/0xf3 [<ffffffff811b7071>] kvasprintf+0x41/0x68 [<ffffffff811b70d6>] kasprintf+0x3e/0x40 [<ffffffff8134d624>] xen_smp_intr_init+0xc8/0x271 [<ffffffff81342c28>] xen_cpu_up+0x315/0x367 [<ffffffff81348662>] _cpu_up+0xb8/0x124 [<ffffffff81348792>] cpu_up+0xc4/0xd0 [<ffffffff8169dca6>] smp_init+0x4a/0x75 [<ffffffff81689d55>] kernel_init_freeable+0x78/0x18d [<ffffffff8133d92a>] kernel_init+0x0/0xd1 [<ffffffff8133d92f>] kernel_init+0x5/0xd1 [<ffffffff81354efc>] ret_from_fork+0x7c/0xb0 [<ffffffff8133d92a>] kernel_init+0x0/0xd1 [<ffffffffffffffff>] 0xffffffffffffffff unreferenced object 0xffff88007c5276d8 (size 16): comm "swapper/0", pid 1, jiffies 4294892303 (age 150.680s) hex dump (first 16 bytes): 63 61 6c 6c 66 75 6e 63 73 69 6e 67 6c 65 32 00 callfuncsingle2. backtrace: [<ffffffff811b70d6>] kasprintf+0x3e/0x40 [<ffffffff810e296d>] __kmalloc_track_caller+0xe7/0xf3 [<ffffffff811b7071>] kvasprintf+0x41/0x68 [<ffffffff811b70d6>] kasprintf+0x3e/0x40 [<ffffffff812126ef>] bind_virq_to_irqhandler+0x3c/0x58 [<ffffffff8134d66d>] xen_smp_intr_init+0x111/0x271 [<ffffffff81342c28>] xen_cpu_up+0x315/0x367 [<ffffffff81348662>] _cpu_up+0xb8/0x124 [<ffffffff81348792>] cpu_up+0xc4/0xd0 [<ffffffff8169dca6>] smp_init+0x4a/0x75 [<ffffffff81689d55>] kernel_init_freeable+0x78/0x18d [<ffffffff8133d92a>] kernel_init+0x0/0xd1 [<ffffffff8133d92f>] kernel_init+0x5/0xd1 [<ffffffff81354efc>] ret_from_fork+0x7c/0xb0 [<ffffffff8133d92a>] kernel_init+0x0/0xd1 [<ffffffffffffffff>] 0xffffffffffffffff unreferenced object 0xffff88007c526810 (size 16): comm "swapper/0", pid 1, jiffies 4294892303 (age 150.680s) hex dump (first 16 bytes): 69 72 71 77 6f 72 6b 32 00 6b 6b 6b 6b 6b 6b a5 irqwork2.kkkkkk. backtrace: [<ffffffff811b70d6>] kasprintf+0x3e/0x40 [<ffffffff810e296d>] __kmalloc_track_caller+0xe7/0xf3 [<ffffffff811b7071>] kvasprintf+0x41/0x68 [<ffffffff811b70d6>] kasprintf+0x3e/0x40 [<ffffffff8134d6c6>] xen_smp_intr_init+0x16a/0x271 [<ffffffff81342c28>] xen_cpu_up+0x315/0x367 [<ffffffff81348662>] _cpu_up+0xb8/0x124 [<ffffffff81348792>] cpu_up+0xc4/0xd0 [<ffffffff8169dca6>] smp_init+0x4a/0x75 [<ffffffff81689d55>] kernel_init_freeable+0x78/0x18d [<ffffffff8133d92a>] kernel_init+0x0/0xd1 [<ffffffff8133d92f>] kernel_init+0x5/0xd1 [<ffffffff81354efc>] ret_from_fork+0x7c/0xb0 [<ffffffff8133d92a>] kernel_init+0x0/0xd1 [<ffffffffffffffff>] 0xffffffffffffffff _______________________________________________ Xen-users mailing list Xen-users@xxxxxxxxxxxxx http://lists.xen.org/xen-users
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |