[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-users] dom0 - oom-killer - memory leak somewhere ? [solved]

On 01/26/12 20:09, John Weekes wrote:

Were you ever able to determine what was using the memory? I had several machines at the same location all enter this state this morning at the same time, and I can't find any clues as to what's exhausting the memory, just as you couldn't. The circumstances suggest that some sort of attack or network event was involved, but I'm not seeing significant traffic levels. One of the machines was brand new (very latest Xen 4.1, very latest xen/next-2.6.32) and had no activity otherwise. It's very puzzling.



We managed to resolve this issue this week. Some extended testing allowed us to get back with the following information.

We are running debian, 2.6.32-5-xen-amd64. Not 100% sure which debian patch level.
2.6.32-5-xen-amd64 (unknown@Debian) #1 SMP Thu May 19 01:16:47 UTC 2011

- the problem also occurs when booting without Xen Hypervisor
- the specific conditions that made the leak triggered is explained bellow

- upgrading to the lastest Debian/stable kernel (2.6.32-41squeeze2) seems to fix the bug
2.6.32-5-amd64 (unknown@Debian) #1 SMP Thu Mar 22 17:26:33 UTC 2012
2.6.32-5-xen-amd64 (unknown@Debian) #1 SMP Thu Mar 22 21:14:26 UTC 2012

The bug was due to improper handling (not sure where) of tagged network packet, while linux had no vlan out of it.

Our network configuration :
br142           8000.b499baac423a       no              eth0.142
br173           8000.b499baac423a       no              eth0.173
br21            8000.b499baac423a       no              eth0

But we also have the VLAN 12 with traffic that might arrive to the box.

If we add eth0.12 (via the following configuration), the leak doesn't hit :

auto br12
iface br12 inet manual
        bridge_ports eth0.12
        bridge_maxwait 0
        bridge_fd 0

upgrade the kernel

Supposed work-around, if upgrading is not an easy option :
configure the switch not to send VLAN not known/configured in the kernel (alternatively, configure the kernel with the specific vlans)


Adrien URBAN, Expert Systèmes - Réseaux - Sécurité - Responsable SN3
www.nbs-system.com, 140 Bd Haussmann, 75008 Paris
Std: +33 158 566 080 / S.Tech: +33 158 566 088 / Fax: +33 158 566 081
Bargento 2012, le 29 mai 2012 au CNIT : www.bargento.com

Xen-users mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.