[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: [Xen-users] Best Practices for PV Disk IO?

  • To: "Christopher Chen" <muffaleta@xxxxxxxxx>, <xen-users@xxxxxxxxxxxxxxxxxxx>
  • From: Jeff Sturm <jeff.sturm@xxxxxxxxxx>
  • Date: Mon, 20 Jul 2009 22:25:18 -0400
  • Cc:
  • Delivery-date: Mon, 20 Jul 2009 19:27:46 -0700
  • List-id: Xen user discussion <xen-users.lists.xensource.com>
  • Thread-index: AcoJmdQiwqX+ewocSFWl5qIewDfSGAAB5o7A
  • Thread-topic: [Xen-users] Best Practices for PV Disk IO?

> -----Original Message-----
> From: xen-users-bounces@xxxxxxxxxxxxxxxxxxx [mailto:xen-users-
> bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Christopher Chen
> Sent: Monday, July 20, 2009 8:26 PM
> To: xen-users@xxxxxxxxxxxxxxxxxxx
> Subject: [Xen-users] Best Practices for PV Disk IO?
> I was wondering if anyone's compiled a list of places to look to
> reduce Disk IO Latency for Xen PV DomUs. I've gotten reasonably
> acceptable performance from my setup (Dom0 as a iSCSI initiator,
> providing phy volumes to DomUs), at about 45MB/sec writes, and
> 80MB/sec reads (this is to a IET target running in blockio mode).

For domU hosts, xenblk over phy: is the best I've found.  I can get
166MB/s read performance from domU with O_DIRECT and 1024k blocks.

Smaller block sizes yield progressively lower throughput, presumably due
to read latency:

256k: 131MB/s
64k:    71MB/s
16k:    33MB/s
4k:     10MB/s

Running the same tests on dom0 against the same block device yields only
slightly faster throughput.

If there's any additional magic to boost disk I/O under Xen, I'd like to
hear it too.  I also pin my dom0 to an unused CPU so it is always
available.  My shared block storage runs the AoE protocol over a pair of
1GbE links.

The good news is that there doesn't seem to be much I/O penalty imposed
by the hypervisor, so the domU hosts typically enjoy better disk I/O
than an inexpensive server with a pair of SATA disks, at far less cost
than the interconnects needed to couple a high-performance SAN to many
individual hosts.  Overall, the performance seems like a win for Xen


Xen-users mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.