[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] [RFC PATCHv2 0/3]: x86/ept: reduce translation invalidation impact

> From: David Vrabel [mailto:david.vrabel@xxxxxxxxxx]
> Sent: Saturday, November 14, 2015 2:50 AM
> This RFC series improves the performance of EPT by reducing the impact
> of the translation invalidations (ept_sync_domain()).  Two approaches
> are used:
> a) Removing unnecessary invalidations after fixing misconfigured
>    entries (after a type change).
> b) Deferring invalidations until the p2m write lock is released.

Do you have a sense which one above incurs more overhead?

> Prior to this change a 16 VCPU guest could not be successfully
> migrated on an (admittedly slow) 160 PCPU box because the p2m write
> lock was held for such extended periods of time.  This starved the
> read lock needed (by the toolstack) to map the domain's memory,
> triggering the watchdog.
> After this change a 64 VCPU guest could be successfully migrated.
> ept_sync_domain() is very expensive because:
> a) it uses on_selected_cpus() and the IPI cost can be particularly
>    high for a multi-socket machine.
> b) on_selected_cpus() is serialized by its own spin lock.
> On this particular box, ept_sync_domain() could take ~3-5 ms.
> Simply using a fair rw lock was not sufficient to resolve this (but it
> was an improvement) as the cost of the ept_sync_domain calls() was
> still delaying the read locks enough for the watchdog to trigger (the
> toolstack maps a batch of 1024 GFNs at a time, which means trying to
> acquire the p2m read lock 1024 times).
> Changes in v2:
> - Use a per-p2m (not per-CPU) list for page table pages to be freed.
> - Hold the write lock while updating the synced_mask.
> David

Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.