Re: [Xen-devel] [PATCH v3] new config option vtsc_tolerance_khz to avoid TSC emulation

[Andrew Cooper <andrew.cooper3@xxxxxxxxxx>]

On 06/03/18 10:41, Olaf Hering wrote:
> Add an option to control when vTSC emulation will be activated for a
> domU with tsc_mode=default. Without such option each TSC access from
> domU will be emulated, which causes a significant perfomance drop for
> workloads that make use of rdtsc.
>
> Add a new domctl XEN_DOMCTL_set_vtsc_tolerance_khz to adjust the
> tolerance value of a running domU that is supposed to be migrated.

Please can we not proliferate the domctls.

This looks like it should be part of the set_tsc_info hypercall, not a
separate hypercall.

>
> One option to avoid the TSC option is to run domUs with tsc_mode=native.
> This has the drawback that migrating a domU from a "2.3GHz" class host
> to a "2.4GHz" class host may change the rate at wich the TSC counter
> increases, the domU may not be prepared for that.

Given that you identify this specifically, why do you think this change
is safe or sensible in the first place?  If using the options here, time
will definitely drift in the VM after migrate.

>
> With this option the host admin can decide how a domU should behave when
> it is migrated across systems of the same class. Since there is always
> some jitter when Xen calibrates the cpu_khz value, all hosts of the same
> class will most likely have slightly different values. As a result vTSC
> emulation is unavoidable. Data collected during the incident which
> triggered this change showed a jitter of up to 200 KHz across systems of
> the same class.

We should see about using better sources of information.  For one, many
Intel systems actually expose the TSC frequency in the bottom of the
PLATFORM_INFO MSR, although this isn't architectural, and has been
replaced with CPUID information in Skylake.

>
> A new utility is added which allows to adjust the vtsc_tolerance_khz
> value for running domUs. This is useful to avoid emulation for domUs
> that are already running and which can not be restarted.
>
> The ordering of records sent during migration is important. The value of
> vtsc_tolerance_khz must be known by the receiving host before
> configuring TSC, because this is the place where the decision of vTSC
> emulation is made. Therefore the existing write_tsc_info function is
> modified to enforce that ordering.

Right, in which case this must be part of set_tsc_info and the TSC
record.  We already have too many problems with co-dependent ordering,
and I'm trying to remove them.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel