Re: [Xen-devel] [PATCH] Add a timer mode that disables pending missed ticks

[Dave Winchell <dwinchell@xxxxxxxxxxxxxxx>]

Hi Keir,

I've added comments below.
See my next mail on some interesting performance numbers.

thanks,
Dave

Keir Fraser wrote:

> On 7/11/07 19:38, "Dave Winchell" <dwinchell@xxxxxxxxxxxxxxx> wrote:
>
>  
>
> > My feeling is that we should go full SYNC. Yes, in theory the
> > guests should be able to handle ASYNC, but in reality it appears that
> > some do not. Since it is easy for us to give them SYNC,
> > lets just do it and not stress them out.
> >    
> One problem with pure SYNC is there's a fair chance you won't deliver any
> ticks at all for a long time, if the guest only runs in short bursts (e.g.,
> I/O bound) and happens not to be running on any tick boundary. I'm not sure
> how much that matters. It could cause time goes backwards if the time
> extrapolation via the TSC is not perfectly accurate, or cause problems if
> there are any assumptions that TSC delta since last tick fits in 32 bits
> (less likely in x64 code I suppose). Anyway, my point is that only testing
> VCPUs under full load may cause us to optimise in ways that have nasty
> unexpected effects for other workloads.
>  
I agree that this could be a problem. I have an idea that could give us full
SYNC and eliminate the long periods without clock interrupts.
In pt_process_missed_ticks() when missed_ticks > 0 set pt->run_timer = 1.
In pt_save_timer():

   list_for_each_entry ( pt, head, list )
       if(!pt->run_timer)
            stop_timer(&pt->timer);

And in pt_timer_fn():

   pt->run_timer = 0;

So, for a guest that misses a tick, we will interrupt him once from the
descheduled state and then leave him alone in the descheduled state.

>  
>
> > For default mode as checked into unstable is now,
> > 64 bit guests should run quite fast as missed is calculated and then a bunch
> > of additional interrupts are delivered. On the other hand
> > 32bit guests very well in default mode.
> >
> > For the original code, before we put in the constant tsc offset business,
> > 64bit guests run poorly and 32bit quests very well time-wise.
> >    
> The default mode hasn't changed. Are you under the impression that
> missed-ticks-but-no-delay-of-tsc is the default mode now? I know x64 guests
> run badly with that because they treat every one of the missed ticks they
> receive as a full tick.
>  
Sorry, I was confused.
However, the default mode will still run poorly for 64 bit guests because
of the pending_nr's accumulated while the guest has interrupts disabled.
As I recall, the effect is quite large, on the order of 10% error.
I'll get you a number later today.

> -- Keir
>
>  
>
> > > Or is the lack of
> > > synchronization of TSCs across VCPUs causing issues that you're trying to
> > > avoid?
> > >      
> > This does cause issues, but its not the only contributor to poor timing.
> > Having TSCs synchronized across vcpus will help some of the time going
> > backwards problems we have seen, I think.
> >
> > Regards,
> > Dave
> >
> > Keir Fraser wrote:
> >
> >    
> >
> > > On 7/11/07 17:29, "Keir Fraser" <Keir.Fraser@xxxxxxxxxxxx> wrote:
> > >
> > >
> > >
> > >      
> > >
> > > > So, you can see we send an interrupt immediately (and ASYNC) if any ticks
> > > > have been missed, but then successive ticks are delivered 'on the beat'. A
> > > > possible middleground? Or perhaps we should just go with SYNC after all...
> > > >   
> > > >
> > > >        
> > > How do these Linux x64 guests fare with the original and default timer mode,
> > > by the way? I would expect that time should be accounted pretty accurately
> > > in that mode, albeit with more interrupts than you'd like. Or is the lack of
> > > synchronisation of TSCs across VCPUs causing issues that you're trying to
> > > avoid?
> > >
> > > -- Keir
> > >
> > >
> > >
> > >
> > >      
>
>  

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel