[Xen-devel] [PATCH][RFC]Remove lock on first guest table walk

["Tian, Kevin" <kevin.tian@xxxxxxxxx>]

Just found this patch in house which I thought sent out
already. :-)

Still one thing remaining is to special case for UP guest
which suffers overhead from this change...

----
Remove lock on first guest table walk

Existing shadow fault path grabs big lock before walking
guest tables, to ensure consistency with shadow content
lest concurrent change from other vcpu in a bad OS.
But this lock brings more lock contention when scaled up
for a good guest which already prevents above case happen.
So this patch tries to remove the lock on first guest
table walk, and then delay check at some special points.

The key is to check whether any guest table update happens
between 1st walk and holding shadow lock. Here we take
two hints for guest table update:
    * write permission removal
    * write emulation
If any above two operations are observed within the race
window, it indicates possiblity that previous walk result
may be inaccurate and re-check is requried. If mismatch,
simply return to trigger another fault.

I made some experiment to sample perfc count for kernel
compile:
<64bit guest>
3.7% of gwalks are re-checked
For re-check, 68% comes from write permission removal
<32bit pae guest>
7.2% of gwalks are re-checked
For re-check, 54.9% comes from write permission removal

Actually previous fast emulation optimization already skip
lots of guest table walks, and thus above ratio can be
smaller if compared to total shadow fault count.

Basically shadow promotion with write permision removal
does suffer higher overhead, but the benefit to reduce
lock contention is more obvious.

Improvement on kernel compile for this patch is:
(64bit Xen, with 2 vcpus guest and 2 physical cpus)
32bit guest: 1.1%
pae guest:   0.4%
64bit guest: 0.5%

Signed-off-by Kevin Tian <kevin.tian@xxxxxxxxx>

Thanks,
Kevin

Attachment: gwalk_no_lock.patch
Description: gwalk_no_lock.patch

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel