RE: vPT rework (and timer mode)

[Paul Durrant <xadimgnik@xxxxxxxxx>]

> -----Original Message-----
> From: Roger Pau Monné <roger.pau@xxxxxxxxxx>
> Sent: 21 July 2020 12:53
> To: paul@xxxxxxx
> Cc: 'Andrew Cooper' <andrew.cooper3@xxxxxxxxxx>; 'Jan Beulich' 
> <jbeulich@xxxxxxxx>; xen-
> devel@xxxxxxxxxxxxxxxxxxxx; 'Wei Liu' <wl@xxxxxxx>; Igor Druzhinin 
> <igor.druzhinin@xxxxxxxxxx>
> Subject: Re: vPT rework (and timer mode)
> 
> On Mon, Jul 06, 2020 at 09:58:53AM +0100, Paul Durrant wrote:
> > > -----Original Message-----
> > > From: Roger Pau Monné <roger.pau@xxxxxxxxxx>
> > > Sent: 06 July 2020 09:32
> > > To: paul@xxxxxxx
> > > Cc: 'Andrew Cooper' <andrew.cooper3@xxxxxxxxxx>; 'Jan Beulich' 
> > > <jbeulich@xxxxxxxx>; xen-
> > > devel@xxxxxxxxxxxxxxxxxxxx; 'Wei Liu' <wl@xxxxxxx>
> > > Subject: Re: vPT rework (and timer mode)
> > >
> > > On Mon, Jul 06, 2020 at 08:03:50AM +0100, Paul Durrant wrote:
> > > > > -----Original Message-----
> > > > > From: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
> > > > > Sent: 03 July 2020 16:03
> > > > > To: Jan Beulich <jbeulich@xxxxxxxx>; Roger Pau Monné 
> > > > > <roger.pau@xxxxxxxxxx>
> > > > > Cc: xen-devel@xxxxxxxxxxxxxxxxxxxx; Wei Liu <wl@xxxxxxx>; Paul 
> > > > > Durrant <paul@xxxxxxx>
> > > > > Subject: Re: vPT rework (and timer mode)
> > > > >
> > > > > On 03/07/2020 15:50, Jan Beulich wrote:
> > > > > > On 01.07.2020 11:02, Roger Pau Monné wrote:
> > > > > >> It's my understanding that the purpose of pt_update_irq and
> > > > > >> pt_intr_post is to attempt to implement the "delay for missed 
> > > > > >> ticks"
> > > > > >> mode, where Xen will accumulate timer interrupts if they cannot be
> > > > > >> injected. As shown by the patch above, this is all broken when the
> > > > > >> timer is added to a vCPU (pt->vcpu) different than the actual 
> > > > > >> target
> > > > > >> vCPU where the interrupt gets delivered (note this can also be a 
> > > > > >> list
> > > > > >> of vCPUs if routed from the IO-APIC using Fixed mode).
> > > > > >>
> > > > > >> I'm at lost at how to fix this so that virtual timers work properly
> > > > > >> and we also keep the "delay for missed ticks" mode without doing a
> > > > > >> massive rework and somehow keeping track of where injected 
> > > > > >> interrupts
> > > > > >> originated, which seems an overly complicated solution.
> > > > > >>
> > > > > >> My proposal hence would be to completely remove the timer_mode, and
> > > > > >> just treat virtual timer interrupts as other interrupts, ie: they 
> > > > > >> will
> > > > > >> be injected from the callback (pt_timer_fn) and the vCPU(s) would 
> > > > > >> be
> > > > > >> kicked. Whether interrupts would get lost (ie: injected when a
> > > > > >> previous one is still pending) depends on the contention on the
> > > > > >> system. I'm not aware of any current OS that uses timer interrupts 
> > > > > >> as
> > > > > >> a way to track time. I think current OSes know the differences 
> > > > > >> between
> > > > > >> a timer counter and an event timer, and will use them 
> > > > > >> appropriately.
> > > > > > Fundamentally - why not, the more that this promises to be a
> > > > > > simplification. The question we need to answer up front is whether
> > > > > > we're happy to possibly break old OSes (presumably ones no-one
> > > > > > ought to be using anymore these days, due to their support life
> > > > > > cycles long having ended).
> > > > >
> > > > > The various timer modes were all compatibility, and IIRC, mostly for
> > > > > Windows XP and older which told time by counting the number of timer
> > > > > interrupts.
> > > > >
> > > > > Paul - you might remember better than me?
> > > >
> > > > I think it is only quite recently that Windows has started favouring 
> > > > enlightened time sources
> rather
> > > than counting ticks but an admin may still turn all the viridian 
> > > enlightenments off so just
> dropping
> > > ticks will probably still cause time to drift backwards.
> > >
> > > Even when not using the viridian enlightenments, Windows should rely
> > > on emulated time counters (or the TSC) rather than counting ticks?
> >
> > Microsoft implementations... sensible... two different things.
> >
> > >
> > > I guess I could give it a try with one of the emulated Windows versions
> > > that we test on osstest.
> > >
> >
> > Pick an old-ish version. I think osstest has copy of Windows 7.
> 
> Tried on Windows 7 (with viridian disabled) setting
> timer_mode="one_missed_tick_pending" and limiting the capacity of the
> domain to 1 (1% CPU utilization) in order to start missing ticks, and
> the clock does indeed start lagging behind.
> 
> When not using one_missed_tick_pending mode and limiting the capacity
> to 1 the clock also lags a bit (I guess with 1% CPU utilization
> delayed ticks accumulate too much), but the clock doesn't seem to be
> skewed that much.
> 
> Both modes will catch up at some point, I assume Windows does sync time
> periodically with the wallclock, but I don't think we want to resort
> to that.
> 

IIRC it normally syncs once an hour or thereabouts. PV drivers will force a 
re-sync every 10 mins if they are installed.

> I will draft a plan about how to proceed in order to fix the emulated
> timers event delivery while keeping the accumulated ticks mode and
> send it to the list, as I would like to fix this.

Ok.

Cheers,

  Paul

> 
> Roger.