Re: [Xen-users] Xen Disk I/O performance vs native performance

[Sami Dalouche <skoobi@xxxxxxx>]

Hi,

1) this is comparing Xen dom0 with native linux running on the same
partition / LVM volume on the same machine

I did a simple dd : 
dd if=/dev/zero of=./t bs=1k count=1048576

Dom0 : 
1073741824 bytes (1.1 GB) copied, 9.21037 seconds, 117 MB/s

DomU : 
1073741824 bytes (1.1 GB) copied, 17.2537 seconds, 62.2 MB/s

The writes are done on the exact same partition, and is consistent (did
it several times). (the 117MB is achieved by shutting down the DomU,
mounting its partition, and running the dd.) the 62 MB is achieved
directly from the DomU..

So, it's not exactly 3 times slower for this case, but I'm pretty sure
that for real applications, lots of small writes or reads, etc, the
behavior is just worse.. so it's definitely between 2 and 3 times slower
on the DomU than on the Dom0.

3] Raid 5 is implemented in hardware, using a 3DM card. I have 8 disks
that are configured in RAID 5 using auto-carving (so it makes 2 physical
disks for the system, which are LVM'ed together). 
The whole LVM disk space is then split into logical volumes, and I have
one LV per VM. 

so for instance, I have a LV : 
/dev/system/blueedge_root

which is referenced as :
kernel = "/boot/vmlinuz-2.6.22-14-xen"
ramdisk = "/boot/initrd.img-2.6.22-14-xen"
builder='linux'
memory = 512 
name = "blueedge"
vcpus = 0
vif = ['mac=02:00:00:00:00:01, bridge=xenbr0' ]
disk = [ 'phy:/dev/mapper/system-blueedge_root,hda1,w']
root = "/dev/hda1 ro"
extra = 'console=xvc0'
#extra='xenconsole=tty'
extra='xencons=tty'

4] nothing interesting in dmesg. and these performance problems have
been consistent over the past few months...

I'll try your VMKNOPPIX thing next time I physically go to the server. I
should be able to post results about this later this week-end.

thanks a lot for your help,
regards,
Sami Dalouche

On Thu, 2008-01-24 at 22:54 +0000, Mark Williamson wrote:
> > Can you guys tell me how much slower your Disk I/O performance is using
> > Xen, compared to native performance ?
> >
> > On my RAID 5 machine, using Ubuntu feisty's
> > linux-ubuntu-modules-2.6.22-14-xen kernel, xen 3.1, LVM disks, read and
> > writes to the disk are something between 2 and 3 times slower.
> >
> > Is that considered normal, or especially bad ? If it's worse than the
> > average, what kind of information do you want me to give you to diagnose
> > the problem ?
> 
> That's lower than I expected.  Can you fill in a few more details about the 
> precise nature of your setup?  The following questions are a starting point:
> 
> 1) Is this comparing Xen dom0 with native Linux running on the same 
> partition / LVM volume on the same machine?
> 2) Are your numbers from the domU or from dom0?
> 3) How are the RAID 5 and LVM implemented?  RAID 5 in hardware or software?  
> LVM in dom0 or domU (or both)?
> 4) Any interesting messages in dmesg that might suggest that the hardware 
> support is not working correctly?
> 5) Any other ideas?
> 
> Another thing that you could try is to download VMKNOPPIX and boot into Xen 
> off that, then try doing some runs mounting your disk and reading / writing 
> it using that.  You might need to rerun this a few times to ensure that 
> everything is paged in off the CD-ROM drive, otherwise that might affect the 
> results badly.  I assume VMKNOPPIX uses something closer to the standard 
> XenLinux kernel, so this might help to eliminate anything Ubuntu's packaging 
> or choice of kernel version may have introduced.
> 
> Cheers,
> Mark
> 
> 


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users