Re: [Xen-devel] Re: Xen & I/O in clusters - problems!

[Håvard Bjerke <Havard.Bjerke@xxxxxxxxxxx>]

On Fri, Oct 15, 2004 at 02:02:31PM +0000, Mark A. Williamson wrote:
> > Between two nodes running plain redhat EL3 with kernel 2.4.21-15.EL:
> >
> >  786.034 MByte/s
> >
> > Between two nodes each running only xen domain 0:
> >
> >  56.480 MByte/s
> 
> That's surprising - I'd have expected any performance problems to involve 
> unpriv domains somehow.  We've never had any performance problems when just 
> running domain 0, even when the code was still under development...
> 

What's possibly even more funny is that when I do the same benchmark localhost 
<-> localhost, ie. through the loopback interface, on domain 0, the bandwidth 
is halved. CPU use is ~100% during both these benchmarks on domain 0 (50% per 
process in the last benchmark). This indicates to me that the bandwidth depends 
on CPU resources. Some heavy processing is happening somewhere.

I suspect this might have something to do with the MPI library scaMPI, which is 
supposed to be more closely linked with the lower layers of the OSI protocol 
stack or something. I will investigate it further.

Another thing that might be worth noting is that we're using the 2.6.8.1 kernel 
on domain 0 as opposed to a 2.4 kernel. I don't think that would make any 
difference, though (except for the fact that modules aren't loaded, but I don't 
think we need any of them anyway).

> It'd be interesting to see your config file (I know it's just the default but 
> it'd be interesting for comparison as there's no obvious reason for your 
> problems).
> 

Attached

Cheers,
Havard

Attachment: config-2.6.8.1-xen0
Description: Text document