Re: [Xen-users] Block device caching causes out of memory in dom0

[Nick Craig-Wood <nick@xxxxxxxxxxxxxx>]

On Fri, Aug 19, 2005 at 04:17:33PM +0100, Ian Pratt wrote:
> > We run our dom0's with 128MB of RAM using xen 2.0.6, kernel 
> > 2.6.11 for dom0 and domUs, and typically have ~10 domUs per 
> > server.  All the domU filingsystems are loopback files served 
> > from dom0 using the file method.
> 
> What are you using to back the loop devices? ext3 on local disk?

ext3 on raid1 local disk

> I'm afraid this is a Linux loopback driver issue. It doesn't provide
> back pressure if memory starts getting very dirty. Its actually quite
> hard for it to do as the memory management system can make it hard for
> it to tell.

Yes thats exactly what it looks like.

I did a bit of experimentation with a loopback file mounted domU vs a
real partition domU.  The loopback file mounted domU exhibits the
problem, but the real partition one doesn't.

So it appears to be the loopback driver caching. Ie the /dev/loop ->
file bit rather than the xen -> /dev/loop bit.

I tried a hacking losetup to open the file in O_DIRECT mode, but
unfortunately it makes no difference - all the magic is done in the
kernel anyway :-(

I also had a look at drivers/block/loop.c to see if there was any way
to hack that to make the loop device unbuffered.  This looks plausible
at least but I'm not familiar enough with kernel internals to figure
it out.

> > Would this be less of a problem if we switched over to using 
> > partitions (lvm say)?  We are slightly reluctant to do this 
> > due to previous bad experiences with lvm though!
> 
> LVM avoids the problem.

Noted - thank you.

> We're looking at our own alternatives to the loop driver.

I'm all ears ;-)

Thanks

Nick
-- 
Nick Craig-Wood <nick@xxxxxxxxxxxxxx> -- http://www.craig-wood.com/nick

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users