Re: [Xen-users] Disk iops performance scalability

[Adi Kriegisch <adi@xxxxxxxxxxxxxxx>]

Hi!

> Seeing a drop-off in iops when more vcpu's are added:-
> 
> 3.8.2 kernel/xen-4.2.1/single domU/LVM backend/8GB RAM domU/2GB RAM dom0
> 
> dom0_max_vcpus=2 dom0_vcpus_pin
> 
> domU  8 cores  fio result 145k iops
[...]
> domU 14 cores fio result 81k iops
[...]
> The initial ioping response is good, then a lot of latency with later ones. 
> Any ideas ?
There are several possible optimizations:
* At first I'd try to set the I/O-Scheduler for all DomUs to 'noop'; there is
  no benefit in scheduling I/Os twice.
* Second thing I'd try is to give Dom0 more scheduler weight
  (xm sched-credit -d 0 -w 512).
Now do the benchmarks again and try to find the bottlenecks by using
dstat/iostat/... to find out what is going on. Probably rebalancing
interrupts on the Dom0 may help (either manual or with the help of
irqbalance).
There are several other optimizations possible, but most of them depend on
your usecase (like increasing RAM on Dom0 for caching, changeing readahead
settings in Dom0 for the storage backend to better fit the I/O requests or
adding more CPUs to Dom0 to be able to better handle the storage backend
and things like that.)

Please post your results!

-- Adi

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users