[Xen-devel] Xend crashes, how to debug?

[Dennis Krul <dweazle@xxxxxxxxx>]

Hi,

We have a very annoying issue we've been unable to iron out. I've googled and browsed through the complete xen-devel and xen-user archives, but it seems we're the only ones having this issue, although I somehow doubt we really are.

Once in a while xend simply crashes. When we don't touch anything it will keep running without problems. But sometimes, usually when we run 'xm create', xend crashes, leaving a 'Domain-Unnamed' behind, which we then have to destroy manually. Quite annoying, especially when your provisioning runs completely unattended.

We don't have any idea where to look. The xen logs are completely useless, no clue what could be wrong whatsoever. A strace didn't provide much useful info and nothing related in the output of 'xm dmesg' either.

I suspect the problem is caused by an I/O bottleneck. I noticed that
ever since I moved the xenstore to a ramdisk (tmpfs), xend crashes less often,
but it still happens.. If our I/O bottleneck is indeed the problem, how
can I verify that? And shouldn't xend be more resilient against these
types of issues? Have there been any patches in xend related to such
issues? Can I increase the verbosity of xend logging perhaps?

Some background info: We run Xen 3.3.2 on Fedora 11 with Linux 2.6.30.7 (Fedora 11 kernel with forward ported Xen patches from OpenSUSE). Maybe it's an unusual setup, but apart from this issue it's actually perfectly stable. We didn't have much luck with Xen 3.4, so we decided to stick with 3.3.2 for now. The dom0 has 1Gb memory, which should be enough (and most of it is unused). 

Can anyone point us into the right direction on how to debug this issue? I don't have much knowledge about xen internals, so I'd appreciate any pointers. Thanks! :)

-- 
Dennis Krul <dweazle@xxxxxxxxx>

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel