Xen project Mailing List

Hi there

I am working with different Xen versions (3.4.0 to 4.1.3) from different vendors (Oracle OVM, Citrix Xen) and observed a strange behavior.

When I try to create some blocks with a bigger blocksize like 1024kbyte the blocks are always split into smaller blocks with a maximum of 44kbytes. I tried this on many different systems and mostly got the same result.

The average queue size of an VM, visible in sar (sar -d), is always 85-88 (*512/1024 = 42.5-44KByte). On the Dom0 it is mostly the same...So every 1024KB block is split up into +/- 23 smaller blocks. Qlogic normally recommends an frame size of 2048kb which is waaaay bigger then what I can create here from an VM....

Example:

dd if=/dev/xvda of=/dev/null iflag=direct bs=1024k

Monitored with:
sar -d 1 200

Example output:
06:59:46 AM DEV tps rd_sec/s wr_sec/s avgrq-sz avgqu-sz await svctm %util
06:59:47 AM dev202-1 1968.69 168008.08 0.00 85.34 15.67 7.93 0.51 101.01

I created an Excel on Google Documents with my results:
https://docs.google.com/spreadsheet/ccc?key=0AhPc4GvvzSpxdGRnZm1WZTZkUXFKYm5JcFZXMjM2MEE&usp=sharing

Please feel free to reproduce the test on your system and put the results into the document.

When I do the same test directly on the Dom0 on the same device I got a blocksize up to 512 kbyte but not higher which is still better then the 44kbyte I got from within the VM....

Has anyone an idea whats is going on here and if its possible to tweak that somehow?

I tested this also with amazon cloud and got the same result of 44kbyte.
On KVM the limit looks to be 128kbyte.
On the commercial Xen Citrix I got 44kbyte within the VM but the sar on the Dom0 showed me 256kbyte (also the SAN System reported 256kbyte). So there it looks like they would do some kind of a tweak, but I discovered this only on the Commercial Citrix Xen...

I would be very happy for some help or hints on this topic

Thank you
Michale Wirz
Switzerland

[Xen-users] Average Request Size in Xen (avgrq-sz)