[Xen-devel] Poor write-performance with nfs

[Jens Krehbiel-GrÃther <jens.krehbiel-graether@xxxxxxxxxxxxxxxxxx>]

Hi!

I have a question about nfs-performance. I don't where the problem is.

My setup:

Host dom0 with 8TB harddisk space 8GB RAM and 6 core CPU.
The hard disk raid is fast:

<cut>
root@midas-phalanx:/data# time dd if=/dev/zero of=test.null bs=1M count=1024
1024+0 DatensÃÂtze ein
1024+0 DatensÃÂtze aus
1073741824 Bytes (1,1 GB) kopiert, 3,27874 s, 327 MB/s

real    0m3.303s
user    0m0.004s
sys     0m1.792s
</cut>

so the write performance is very good on harddisk.

I created several domU guests which do a nfs-boot from the local host. 
(strange setup, I know). I don't want to define the size of guests disk 
so I don't want to use a image as hard disk image. This is why I do a 
nfs-boot.
The dom0 exports the file system for the guest. I use network bridging. 
My guest config file is the following:
<cut>
kernel          = '/boot/vmlinuz-2.6.32-5-xen-amd64'
ramdisk         = '/boot/initrd.img-2.6.32-5-xen-amd64'
memory          = 1024
name            = 'ufp-mysql-01'
hostname        = 'ufp-mysql-01'
#dhcp           = 'dhcp'
vif             = [ 'mac=00:16:3e:00:42:06, bridge=lan' ]
interface       = 'eth0'
ip              = '10.10.42.206'
netmask         = '255.255.255.0'
gateway         = '10.10.42.204'
nfs_server      = '10.10.42.108'
nfs_root        = '/data/nfs-root/ufp-mysql-01'
root            = '/dev/nfs'
extra           = '2 console=hvc0 xencons=tty boot=nfs rootfstype=nfs'
</cut>

The guest boots without error. But the write performance over the nfs is 
very poor:
<cut>
root@ufp-mysql-01:~# time dd if=/dev/zero of=test.null bs=1M count=512
512+0 DatensÃtze ein
512+0 DatensÃtze aus
536870912 Bytes (537 MB) kopiert, 2159,08 s, 249 kB/s

real    35m59.133s
user    0m0.000s
sys     0m1.396s
</cut>

The read-performance is good:

<cut>
root@ufp-mysql-01:~# dd if=test.null of=/dev/zero bs=1M
512+0 DatensÃtze ein
512+0 DatensÃtze aus
536870912 Bytes (537 MB) kopiert, 0,705893 s, 761 MB/s
</cut>

But it is not the performance of the server itself:

<cut>
root@ufp-mysql-01:~# dd if=/dev/zero of=/dev/null bs=1M count=512
512+0 DatensÃtze ein
512+0 DatensÃtze aus
536870912 Bytes (537 MB) kopiert, 0,0655885 s, 8,2 GB/s
</cut>

OK, this is the setup where the nfs-server is on dom0 and exports the 
filesystem for domU.
When I copy the rootfs to another computer and export it from there 
(with same export options) I get a lot of more performance.
This is the guest bootet from nfs on other server:

<cut>
root@ufp-mysql-01:~# time dd if=/dev/zero of=test.null bs=1M count=512
512+0 DatensÃtze ein
512+0 DatensÃtze aus
536870912 Bytes (537 MB) kopiert, 15,506 s, 34,6 MB/s

real    0m15.617s
user    0m0.004s
sys     0m1.260s
</cut>

The "external" nfs-server has a single hard disk, so this performance is 
OK for this config.
Can you give me a hint why the perfmance is so bad when I export the fs 
from dom0? How can i solve this?
The server has no I/O jobs while this test was running so the whole 
performance should be available.
Jens


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel