Re: [Xen-devel] NFS related netback hang

[Wei Liu <wei.liu2@xxxxxxxxxx>]

On Fri, Apr 12, 2013 at 11:34:50AM +0100, G.R. wrote:
> On Fri, Apr 12, 2013 at 5:27 PM, Wei Liu <wei.liu2@xxxxxxxxxx> wrote:
> > On Fri, Apr 12, 2013 at 04:10:34AM +0100, G.R. wrote:
> >>
> >> Yes, it's not specific to NFS page, but I'm just bad luck enough.
> >> I agree with your suspect, the chance depends on the memory pressure in 
> >> dom0.
> >> So here is a proper setup to reproduce the issue:
> >> 1. dom0 with SWAP disabled and with limited memory allocated.
> >> 2. domU serves storage and exports NFS
> >> 3. dom0 mounts the domU storage and writes to it.
> >> 4. You need to achieve high speed to expose this issue.
> >>
> >
> > My setup is almost the same. The write speed is around
> > 35-45MB/s if I do:
> >
> > dd if=/dev/zero of=/mnt/t bs=1 count=200
> >

Oh this is in fact bs=1M.

> > However if I do count=2000, the speed slows down to 24MB/s. I suspect
> > that's the memory pressure in Dom0 - my Dom0 only has 1024MB Ram. But I
> > still didn't see any error.
> >
> That's weird, the stack trace can prove that the issue exists. And the
> issue stands theoretically.
> But why this is common in my build and cannot be reproduced in yours?
> There must be some factor got missed here.
> Is there any kernel config affecting the memory management behavior in
> dom0? What's your dom0 kernel version?

I use default memory management options, i.e. I didn't touch any of
those. I use 3.8-rc7.

> Is there anything that could matter in nfs config?

I use following line in /etc/exports:
/ *(rw)

> Do you enable memory ballooning for dom0? I do. But does it matter?

CONFIG_XEN_BALLOON=y

I have my config file attached.

> 
> I still believe the key factor is to stress the memory.
> Maybe you can try further limit the memory size and use a larger file size.
> 
> I become uncertain about how the transfer speed affects.
> I can achieve 10GB/s in iperf test without issue.
> And ftp transfer also works without problem at 50MB/s
> But may be the higher net speed is a negative factor here -- NFS may
> be able to commit changes in faster speed.
> Probably we should feed data faster than NFS can handle so that memory
> is used up quickly?
> But the back pressure from down stream should slow down the speed that
> upstream is eating the memory.
> How does the throttling works? Anyway to control?
> 
> I'll check why my dom0 reported OOM, may be that's one factor too.
> 

This is a good starting point. :-)


Wei.

Attachment: dom0-config
Description: Text document

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel