|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-users] Poor disk io performance in domUs
On Jun 20, 2007, at 10:49 PM, Andrej Radonic wrote:Hi,in a previously well behaving xen setup we are suddenly experiencing massive loss of disk performance.Setup: two identical machines (dell, 2 quadcore cpus each, 8 GB RAM each), attached to *one* SAS storagebox (Raid5 SAS disks), CentOS 5, Xen 3.04 running three domUs each. Storage partitions are exposed as block-devices to the guests. Filesystem is ext3. raid5 ... so you've got the mailserver running on multiple hard drives? Since a domU running a rather heavily loaded mailserver (many small files, many reads/writes) has been added, io performance has deteriorated significantly.In dom0 about 100 MB/s can be achieved, in domU only about 50 MB/s. This looks even worse when there are massive concurrent / parallel io jobs, then throughput goes down to 20 MB/s. That is to be expected. If the drives are busy seeking all over the place, you are _not_ going to get high speed transfers off of them. I don't see anything xen specific about this. Although it is more likely to show up when you put multiple tasks on a single set of hardware (and Xen encourages that). During all this the cpus are mostly idle...Any ideas? I know some performance degradation is to be expected but shurely not on this scale? Try it on a single drive/cpu machine... fire up something like bonnie to get it to do multiple parallel I/Os while timing a big dd run or something in the background. Maxing out your disk drives is the easiest way to drive performance into the ground :(. You can only get 50-100 MB/s transfers off a drive if the data is being read sequentially off the drive with an absolute minimum of head seeks. As soon as something else starts moving the drive head(s) around, your sustained transfer rate is going to fall dramatically... it comes down to how many seeks per second your drives can do. Building up a big RAID array improves the system's ability to provide high throughput but is still limited by the abilities of the drives it is built from.... Mail servers, like news servers, "eat" drives.... If space allows it, you may be better off buying more smaller drives than fewer larger ones. With more drives you have more spindles and the ability to do more seeks per second (and higher combined throughput too). I try to isolate the drives for my domUs... often dedicating one spindle to one domU, and using drbd for raid1. It would be nicer if this wasn't needed (since it is the opposite of virtualization), but AFAIK the one resource that is hard to schedule/limit/allocate/prioritize is disk access/bandwidth. -Tom _______________________________________________ Xen-users mailing list Xen-users@xxxxxxxxxxxxxxxxxxx http://lists.xensource.com/xen-users
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |