[Xen-users] Re: Performance Issues: I/O Wait

["Nick Couchman" <Nick.Couchman@xxxxxxxxx>]

Hi, again...haven't had any responses to this, yet.  I'd really appreciate any 
light that anyone could shed on the issue.  By way of update, it doesn't seem 
to necessarily be the network, it just seems to happen more frequently on 
network-based traffic.  Local block devices experience the same issues, just 
not quite as long sitting and waiting.  I'm open to any suggestions.  I may try 
to set it to a single vCPU and see if that helps, but I can't understand why 
this only happens in the guest and I see absolutely no signs of it in the Dom0. 
 Any suggestions for how to track down why it's happening would also be 
appreciated.

Thanks,
Nick

>>> Nick Couchman 10/18/07 11:05 AM >>>
Hey, everyone, 
I'm having some issues with a Xen DomU right related to performance.  We have 
an application that cross-compiles a Linux distribution from scratch for 
embedded systems.  We're attempting to run this application inside a Xen DomU 
(paravirtualized, modified guest kernel), and the performance is really bad.  
The culprit seems to be high I/O wait times related to the network interface. 

The host machine is a Dell PowerEdge 1950 with 2 x Dual-Core Xeon processors 
(Xeon 5150 @ 2.66GHz, 1333 FSB).  The domU is configured with 2 vCPUs and 1GB 
of RAM.  The Dom0 O/S is SuSE Linux Enterprise Server 10, and the DomU O/S is 
CentOS4U5.  For comparison, we're comparing against a physical machine that has 
a Pentium 4 3.2GHz processor on an 800MGz FSB with 4GB of RAM.  The RAM does 
not seem to be an issue - on the DomU, about half of the 1024MB is used by 
active processes and the other half is left to buffering and caching.  There 
are 4 kbytes of memory used on the swap partition.  Building these Linux 
distributions on the physical system takes 70-80 minutes (real time) - on the 
DomU system it takes 130-140 minutes. 

The few times that I've see the system "lock up" the way the engineers who are 
using it claim it is doing, one of the DomU vCPUs goes to 100% wait for 20-30 
seconds.  This seems to coincide with network operations in the DomU.  So, I'm 
wondering, what can I do to improve network performance and eliminate these I/O 
wait times on this CPU.  The server itself doesn't seem to be having any 
problems - while this occurs in the DomU, other physical systems can access the 
server just fine without any issues.  Any light that anyone can shed on the 
situation - how I can improve network performance and eliminate these I/O waits 
- would be most appreciated.  Here's the network config line from my Xen 
configuration file: 

vif=[ 'mac=00:16:3e:75:0d:be,bridge=xenbr108', ] 

And I found a suggestion for traffic control in the guest that I applied - 
maybe I did something wrong there, too.  Here's the output from tc qdisc show: 
qdisc tbf 8001: dev eth0 rate 50Mbit burst 50Kb lat 20.0ms 

Thanks, 
Nick 

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users