[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-users] Disk-I/O Performance DomU vs Dom0



Hi,
I experience massive I/O performance-difference between Dom0 and DomU. To 
simply show the impact, I use a logical volume striped over 6 hard-drives. Then 
(write/read) enough bytes linear via dd zeros (on/from) this lv and compare the 
time.
The result: inside DomU I get only about 30% of the I/O performance of the Dom0.

So, where is the handbrake, and how do I release it? Normaly there shouldbe no 
such big difference. If you need more information about setup and config, to 
give me a hint where else to search for my missing 70% performance, simply 
reply inside the list.

Here are the gory details so far:

Preparations
==================================================================================================
Dom0: akaar
DomU: guinan

Create LV:
lvcreate --name=performance-test --size=20G --stripes=6 vg0

Attach LV to domU:
root@akaar:~# xm block-attach guinan phy:/dev/vg0/performance-test xvdc1 w

Test:
=====================================================================================================
============== Write
=== Dom0
root@akaar:~# date ; dd if=/dev/zero of=/dev/vg0/performance-test bs=1M 
count=4000 ; sync ; date
Do 5. Apr 20:15:50 CEST 2012
4000+0 records in
4000+0 records out
4194304000 bytes (4,2 GB) copied, 7,34972 s, 571 MB/s
Do 5. Apr 20:15:58 CEST 2012

=== DomU
root@guinan:~# date ; dd if=/dev/zero of=/dev/xvdc1 bs=1M count=4000 ; sync ; 
date
Do 5. Apr 20:34:50 CEST 2012
4000+0 records in
4000+0 records out
4194304000 bytes (4,2 GB) copied, 23,5331 s, 178 MB/s
Do 5. Apr 20:35:13 CEST 2012


============== Read
=== Dom0
root@akaar:~# date ; dd if=/dev/vg0/performance-test of=/dev/null bs=1M 
count=4000 ; sync ; date
Do 5. Apr 20:21:38 CEST 2012
4000+0 records in
4000+0 records out
4194304000 bytes (4,2 GB) copied, 8,98257 s, 467 MB/s
Do 5. Apr 20:21:48 CEST 2012

=== DomU
root@guinan:~# date ; dd if=/dev/xvdc1 of=/dev/null bs=1M count=4000 ; sync ; 
date
Do 5. Apr 20:38:04 CEST 2012
4000+0 records in
4000+0 records out
4194304000 bytes (4,2 GB) copied, 17,8235 s, 235 MB/s
Do 5. Apr 20:38:22 CEST 2012
==================================================================================================
==================================================================================================


==================================================================================================
Technical Data:
===================================================================================================
Hardware Poweredge 510 /  96GB RAM / 2x X5650

Storage 6 Nearline SAS Harddrives attached to Dell Perc H200:

lvcreate --name=performance-test --size=20G --stripes=6 vg0

Dom0
===================================================================================================
 Default Debian Squeeze 32 Bit Kernel 64Bit HV:
1GB Ram / fixed / ballooning disabled / all cpus / no pinning

root@akaar:~# uname -a
Linux akaar 2.6.32-5-xen-686 #1 SMP Mon Mar 26 10:32:49 UTC 2012 i686 GNU/Linux


root@akaar:~# xm info
host                   : akaar
release                : 2.6.32-5-xen-686
version                : #1 SMP Mon Mar 26 10:32:49 UTC 2012
machine                : i686
nr_cpus                : 24
nr_nodes               : 2
cores_per_socket       : 6
threads_per_core       : 2
cpu_mhz                : 2666
hw_caps                : 
bfebfbff:2c100800:00000000:00001f40:029ee3ff:00000000:00000001:00000000
virt_caps              : hvm hvm_directio
total_memory           : 98291
free_memory            : 95010
node_to_cpu            : node0:0,2,4,6,8,10,12,14,16,18,20,22
                         node1:1,3,5,7,9,11,13,15,17,19,21,23
node_to_memory         : node0:47110
                         node1:47899
node_to_dma32_mem      : node0:0
                         node1:2990
max_node_id            : 1
xen_major              : 4
xen_minor              : 0
xen_extra              : .1
xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 
hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler          : credit
xen_pagesize           : 4096
platform_params        : virt_start=0xf9c00000
xen_changeset          : unavailable
xen_commandline        : placeholder dom0_mem=1024M
cc_compiler            : gcc version 4.4.5 (Debian 4.4.5-8)
cc_compile_by          : waldi
cc_compile_domain      : debian.org
cc_compile_date        : Mon Nov  7 09:16:11 CET 2011
xend_config_format     : 4

root@akaar:~# xm list
Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0   997    24     r-----   1080.2
guinan                                       2  4096     4     -b----      0.0

root@akaar:~# xm vcpu-list
Name                                ID  VCPU   CPU State   Time(s) CPU Affinity
Domain-0                             0     0    20   -b-     747.4 any cpu
Domain-0                             0     1    17   -b-      80.2 any cpu
Domain-0                             0     2    12   -b-     105.4 any cpu
Domain-0                             0     3    19   -b-      49.0 any cpu
Domain-0                             0     4     4   -b-      30.7 any cpu
Domain-0                             0     5    13   -b-      35.9 any cpu
Domain-0                             0     6    21   -b-      52.3 any cpu
Domain-0                             0     7    23   -b-      37.9 any cpu
Domain-0                             0     8     2   -b-      26.9 any cpu
Domain-0                             0     9     7   -b-      32.7 any cpu
Domain-0                             0    10    14   -b-      34.8 any cpu
Domain-0                             0    11    16   -b-      26.2 any cpu
Domain-0                             0    12    10   -b-      23.7 any cpu
Domain-0                             0    13     8   -b-      38.7 any cpu
Domain-0                             0    14     5   -b-      34.8 any cpu
Domain-0                             0    15    11   -b-      30.3 any cpu
Domain-0                             0    16     3   -b-      29.3 any cpu
Domain-0                             0    17    18   -b-      41.2 any cpu
Domain-0                             0    18    14   -b-      41.4 any cpu
Domain-0                             0    19     1   -b-      45.6 any cpu
Domain-0                             0    20     6   r--      43.6 any cpu
Domain-0                             0    21    15   -b-      29.2 any cpu
Domain-0                             0    22     0   -b-      23.2 any cpu
Domain-0                             0    23     7   -b-      30.5 any cpu
guinan                               2     0     9   -b-      72.1 any cpu
guinan                               2     1    15   -b-       2.9 any cpu
guinan                               2     2    19   -b-       7.4 any cpu
guinan                               2     3    13   -b-       6.0 any cpu
=========================================================================================================


DomU
=========================================================================================================
debian Squeeze 64Bit 1GB RAM / 4 vcpus / no pinning:

root@guinan:~# uname -a
Linux guinan 2.6.32-5-xen-amd64 #1 SMP Thu Mar 22 21:14:26 UTC 2012 x86_64 
GNU/Linux

root@guinan:~# free
             total       used       free     shared    buffers     cached
Mem:       1049180     227444     821736          0      20060      73712
-/+ buffers/cache:     133672     915508
Swap:      2101176          0    2101176



Attach Storage to DomU guinan
root@akaar:~# xm block-attach guinan phy:/dev/vg0/performance-test xvdc1 w

lvcreate --name=performance-test --size=20G --stripes=6 vg0

Cheers
    Ulli
--
Ulrich Hochholdinger
Bereichsleiter Infrastruktur

Netpioneer GmbH, Beiertheimer Allee 18, 76137 Karlsruhe
Geschäftsführer: Jürgen Diller
HRB: 107521 Handelsregister Mannheim

Tel: 0721 / 9 20 60 90
Fax: 0721 / 9 20 60 30
E-Mail: Ulrich.Hochholdinger@xxxxxxxxxxxxx
www: http://www.netpioneer.de

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.