RE: [Xen-users] Block I/O questions

["Petersson, Mats" <Mats.Petersson@xxxxxxx>]

> -----Original Message-----
> From: xen-users-bounces@xxxxxxxxxxxxxxxxxxx 
> [mailto:xen-users-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of 
> Gerd Stolpmann
> Sent: 29 March 2006 14:10
> To: xen-users@xxxxxxxxxxxxxxxxxxx
> Subject: [Xen-users] Block I/O questions
> 
> Hello,
> 
> after doing a benchmark where Xen 2.0.7 was magnitudes of 
> orders faster than the native system, I have some questions 
> about how Xen performs block I/O.
> 
> The particular benchmark forks 100 processes, and each 
> process writes a file of 50 MB, writes it again, reads it, 
> and reads it again. Because we are doing that 100 times in 
> parallel, I/O scheduling has a massive effect on the results. 
> (See below for the script.)
> 
> For a native system I measured:
> 
> - first write: 500 KB/s throughput per process (but sometimes
>   much less)
> - second write: 460 KB/s
> - first read: 80 KB/s
> - second read: 80 KB/s
> 
> The Xen VM reversed things:
> 
> - first write: 100 KB/s
> - second write: 400 KB/s
> - first read: 410 KB/s
> - second read: 410 KB/s
> 
> As you can see the VM is much faster reading the files. Do 
> you have an explanation for this?

Yes, there's more buffering capacity in the system - Dom0 buffers the
file-system, and then the DomU buffers things on top of that. 
> 
> The other surprising effect is that the first write (where 
> the blocks for the files are allocated) is slower in the VM.

That's probably because you have much more latency here: Creating a file
is many more operations than "re-creating" a file - new directory
entries (inodes) need to be created, etc. 

Since Xen must go through two layers of disk drivers, it's going to be
slower than the Native system. When you do the next set of operations,
the files are just replaced in the directory structure, and much of the
data is cached anyways, so you don't have to wait for the physical
access to happen. 

> 
> I suppose Xen has additional I/O buffers. Can somebody 
> describe how they change I/O in general? Are the I/O requests 
> batched or reordered?

I don't believe there is any (intentional) re-ordering or batching, but
since you have a two-level caching mechanism, you could end up with a
bit more caching and of course, those caches may well (due to being
larger) change the actual order that data/meta-data is ending up on the
physical disk.
> 
> Is there an impact on the safety of the disk data? Does 
> buffering increase the likeliness of filesystem corruption? 
> Are I/O barriers respected?

It shouldn't have any noticable effect other than time-wise. If it does,
then something is broken and should be fixed - but that's just how I see
it, and I'm not quite sure exactly how the disk interface works in this
respect. I certainly haven't seen any increased problems with
FS-corruption on my system, but I haven't exactly tried to stress it in
order to create those... 

--
Mats
> 
> The script uses iozone (www.iozone.org). /Data is an empty 
> ext3 partition stored in a physical partition.
> 
> files=""
> n=0
> while [ $n -lt 100 ]; do
>   files="$files /Data/testfile.$n"
>   n=$((n+1))
> done
> iozone -i 0 -i 1 -r 64k -s 50M -t 100 -e -F $files
> 
> Gerd
> 
> P.S. I also ran this benchmark on ESX server. It crawls...
> 
> --
> ------------------------------------------------------------
> Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
> gerd@xxxxxxxxxxxxxxxxx          http://www.gerd-stolpmann.de
> Phone: +49-6151-153855                  Fax: +49-6151-997714
> ------------------------------------------------------------
> 
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-users
> 
> 


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users