Re: [Xen-devel] PV performance degraded after live migration

["Jan Beulich" <JBeulich@xxxxxxxx>]

>>> On 15.03.17 at 17:41, <boris.ostrovsky@xxxxxxxxxx> wrote:
> On 03/15/2017 12:02 PM, Jan Beulich wrote:
>>>>> On 15.03.17 at 16:26, <andrew.cooper3@xxxxxxxxxx> wrote:
>>> On 15/03/17 15:23, Olaf Hering wrote:
>>>> On Wed, Mar 15, Olaf Hering wrote:
>>>>
>>>>> On Wed, Mar 15, Andrew Cooper wrote:
>>>>>> As a crazy idea, doest this help?
>>>>>> tsc->incarnation = 0
>>>> This does indeed help. One system shows now the results below, which
>>>> means the performance goes down during migration (to localhost) and goes
>>>> back to normal after migration.
>>>>
>>>> What impact has such change to ->incarnation?
>>> So what this means is that, after migrate, Xen sees that the advertised
>>> TSC value doesn't match the current hardwares TSC, so enables rdtsc
>>> interception so the values reported back can be emulated at the old
>>> frequency.
>>>
>>> There is no easy solution to this problem.
>> Especially for localhost migration it ought to be possible to deal
>> with this. What I'm wondering is why the frequency comparison
>> in default mode handling in tsc_set_info() is HVM-only:
>>
>>         if ( tsc_mode == TSC_MODE_DEFAULT && host_tsc_is_safe() &&
>>              (has_hvm_container_domain(d) ?
>>               (d->arch.tsc_khz == cpu_khz ||
>>                hvm_get_tsc_scaling_ratio(d->arch.tsc_khz)) :
>>               incarnation == 0) )
>>         {
>>
>> Boris, you've made it be this way in 82713ec8d2 ("x86: use native
>> RDTSC(P) execution when guest and host frequencies are the
>> same").
> 
> Don't know why.
> 
> In fact, I looked at the history of this patch and earlier versions had
> 
> @@ -1889,10 +1890,14 @@ void tsc_set_info(struct domain *d,
>          d->arch.vtsc_offset = get_s_time() - elapsed_nsec;
>          d->arch.tsc_khz = gtsc_khz ? gtsc_khz : cpu_khz;
>          set_time_scale(&d->arch.vtsc_to_ns, d->arch.tsc_khz * 1000 );
> -        /* use native TSC if initial host has safe TSC, has not migrated
> -         * yet and tsc_khz == cpu_khz */
> -        if ( host_tsc_is_safe() && incarnation == 0 &&
> -                d->arch.tsc_khz == cpu_khz )
> +        /*
> +         * Use native TSC if initial host has safe TSC and either has not
> +         * migrated yet or tsc_khz == cpu_khz (either "naturally" or via
> +         * TSC scaling)
> +         */
> +        if ( host_tsc_is_safe() &&
> +             (incarnation == 0 || d->arch.tsc_khz == cpu_khz ||
> +              cpu_has_tsc_ratio) )
>              d->arch.vtsc = 0;

Which, as pointed out back then, was not PV-correct.

> However, the original code was even stricter.

True.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel