[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-devel] [PATCH] Fix performance problems with mprotect()

While working on a database scaling problem using a SAP database test suite we 
discovered that these enterprise level large shared memory databases are very 
heavy users of mprotect(), to the extent that the performance overhead in 
current Xenolinux  impacts scaling beyond a few cpus quite badly.  A single cpu 
run under Xen has a nominal impact, but scaling out to 8 cpus results in a 
performance of less than 10% of native throughput. Ouch!

The problem was isolated to the relatively high overhead of how mprotect() 
updates ptes, and the lack of efficient batching opportunities as presently 
implemented.  By adding a hypercall which batches pte updates such that 
hardware flags updates (accessed or dirty bits) are not lost, and redoing the 
performance critical portion of mprotect() to take advantage of that, 
performance has improved to where it is essentially equivalent to native 
performance with this SAP database test suite.  I haven't tested other database 
implementations, but I would imagine they would have similar results.

This is obviously quite an improvement and should allow Xen to be deployed for 
more enterprise level workloads.

There are actually 2 new hypercalls included in this patch - one that batches 
flags-only changes to pte's within a single page table, and another which 
allows the full pte to be changed - both "atomically" as described above.

I imagine there are other places in Linux that could benefit from these 
hypercalls, but these patches focus on the mprotect performance problem only.

The hypervisor patch was mostly written and tested with the  3.0.4 era 
hypervisor that ships with SLES 10, and then ported and tested on current 
The Linux patch was written and tested on 2.6.16 and made apply to the 2.6.18 
tree without further testing.

Feedback and comments are appreciated.

Signed-off-by: Bruce Rogers <brogers@xxxxxxxxxx>

- Bruce Rogers

Attachment: xen_mprotect_perf.patch
Description: Text document

Attachment: linux_mprotect_perf.patch
Description: Text document

Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.