[Xen-devel] [PATCH 0/5] x86/time: PVCLOCK_TSC_STABLE_BIT support

[Joao Martins <joao.m.martins@xxxxxxxxxx>]

Hey,

This series is a repost with the comments that I got so far and
hopefully could be considered for Xen 4.7.

PVCLOCK_TSC_STABLE_BIT is the flag telling the guest that the
vcpu_time_info (pvti) are monotonic as seen by any CPU, a feature
which is currently not supported. As it is (i.e. bindly setting the
flag), we can observe that this property isn't there: a process using
vdso clock_gettime/gettimeofday will observe a significant amount of
warps (i.e. time going backwards) and it's due to 1) time calibration
skew in xen rendezvous algorithm 2) clocksource not in sync with TSC.
These warps are seen more frequently on PV guests (potentially because
vcpu time infos are only updated when guest is in kernel mode, and
perhaps lack of tsc_offset?), and in rare ocasions on HVM guests. It
is worth noting that with guests VCPUs pinned, only PV guests see
these warps. But on HVM guests specifically: such warps only occur
when one of guest VCPUs is pinned to CPU0. This series aims to propose
a solution to that and it's divided as following:

(R)     * Patch 1: Adds the missing flag field to vcpu_time_info.
(U)     * Patch 2: Adds a new clocksource based on TSC
(U)     * Patch 3, 4: Adjustments for patch 5
        * Patch 5: Implements the PVCLOCK_TSC_STABLE_BIT

        [R := Reviewed-by ;; U := Updated]

PVCLOCK_TSC_STABLE_BIT is set only when using clocksource=tsc,
and so it remains optional unless specified by the admin.

The test was running time-warp-test, that constantly calls
clock_gettime/gettimeofday on every CPU. It measures a delta with the
previous returned value and mark a warp if it's negative. I measured
it during periods of 1h and 6h and check how many warps and their
values (alongside the amount of time skew). Measurements/Changelog are
included in individual patches.

Note that most of the testing has been done with Linux 4.4 in which
these warps/skew could be easily observed as vdso would use each vCPU
pvti. Though Linux >= 4.5 changes this behaviour and use only the
vCPU0 pvti though still requiring PVCLOCK_TSC_STABLE_BIT flag
support.

Any comments appreciated,

Thanks!
Joao

Joao Martins (5):
  public/xen.h: add flags field to vcpu_time_info
  x86/time: implement tsc as clocksource
  x86/time: streamline platform time init on plt_init()
  x86/time: refactor read_platform_stime()
  x86/time: implement PVCLOCK_TSC_STABLE_BIT

 xen/arch/x86/time.c      | 132 ++++++++++++++++++++++++++++++++++++++++-------
 xen/include/public/xen.h |   6 ++-
 2 files changed, 119 insertions(+), 19 deletions(-)

-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel