[Xen-devel] [PATCH v2 0/6] x86/time: PVCLOCK_TSC_STABLE_BIT support

[Joao Martins <joao.m.martins@xxxxxxxxxx>]

Hey,

This series is the v2 of pvclock TSC series (v1 presented here [0]).

PVCLOCK_TSC_STABLE_BIT is the flag telling the guest that the
vcpu_time_info (pvti) are monotonic as seen by any CPU, a feature
which is currently not supported. As it is (i.e. bindly setting the
flag), we can observe that this property isn't there: a process using
vdso clock_gettime/gettimeofday will observe a significant amount of
warps (i.e. time going backwards) and it's due to 1) time calibration
skew in xen rendezvous algorithm 2) clocksource not in sync with TSC.
These warps are seen more frequently on PV guests (potentially because
vcpu time infos are only updated when guest is in kernel mode, and
perhaps lack of tsc_offset?), and in rare ocasions on HVM guests. It
is worth noting that with guests VCPUs pinned, only PV guests see
these warps. But on HVM guests specifically: such warps only occur
when one of guest VCPUs is pinned to CPU0. This series aims to propose
a solution to that and it's divided as following:

 U      * Patch 1: Adds the missing flag field to vcpu_time_info.
 N      * Patch 2: Small refactor around init_platform_time to reuse
                   initialization code when switching to TSC.
 U      * Patch 3: Adds a new clocksource based on TSC
 R,R    * Patch 4, 5: Adjustments for patch 5
 U      * Patch 5: Implements the PVCLOCK_TSC_STABLE_BIT

        [ R := Reviewed-by ;; U := Updated ;; N := New ]

PVCLOCK_TSC_STABLE_BIT is set only when using clocksource=tsc and CPU
hotplug is not meant to be perfomed, and remains optional unless specified
by the admin. nocpuhotplug is the option telling that no cpu hotplug is
to be perfomed which overrides system default behaviour when initializing
TSC i.e. testing max_vcpus and number of present cpus.

The test was running time-warp-test, that constantly calls
clock_gettime/gettimeofday on every CPU. It measures a delta with the
previous returned value and mark a warp if it's negative. I measured
it during periods of 1h and 6h and check how many warps and their
values (alongside the amount of time skew). Measurements/Changelog are
included in individual patches.

Note that most of the testing has been done with Linux 4.4 in which
these warps/skew could be easily observed as vdso would use each vCPU
pvti. Though Linux >= 4.5 changes this behaviour and use only the
vCPU0 pvti though still requiring PVCLOCK_TSC_STABLE_BIT flag
support.

Thanks!
Joao

[0] 
http://bugs.xenproject.org/xen/mid/%3C1458231136-13457-1-git-send-email-joao.m.martins@xxxxxxxxxx%3E

Joao Martins (6):
  public/xen.h: add flags field to vcpu_time_info
  x86/time: refactor init_platform_time()
  x86/time: implement tsc as clocksource
  x86/time: streamline platform time init on plt_init()
  x86/time: refactor read_platform_stime()
  x86/time: implement PVCLOCK_TSC_STABLE_BIT

 xen/arch/x86/time.c      | 236 +++++++++++++++++++++++++++++++++++++++++------
 xen/include/public/xen.h |   6 +-
 2 files changed, 212 insertions(+), 30 deletions(-)

-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel