RE: [Xen-devel] [PATCH] [RFC] Building guests on monotonic Xen system time

["Dan Magenheimer" <dan.magenheimer@xxxxxxxxxx>]

> Whether that is remove some of the older timer modes now,

Looking at the various timer modes, it appears that:

0==delay_for_missed_ticks is required for non-resilient Linuxes
   (e.g. 32-bit RHEL4-based systems)
1==no_delay_for_missed_ticks is required for Windows(??)
2==no_missed_ticks_pending is required for resilient Linuxes
   (e.g. 64-bit RHEL5-based systems)
3==one_missed_tick_pending DON'T KNOW WHAT IS USED FOR

So timer_mode=3=one_missed_tick_pending might possibly
be retired, but the others couldn't.

Here's another option:  I think delay_for_missed_ticks is
the only mode where time must be set backwards.  All of the
other modes track wallclock.  It looks possible that we
could enforce monotonicity for the other three modes and
not enforce it for delay_for_missed_ticks (which is,
unfortunately, the default).

If this is acceptable to you (and you don't disagree with
my "only mode...set backwards" conjecture), I'll try to
code a patch for this.

Thanks,
Dan

P.S. I think the answer is no, but is there any way to specify
gang-scheduling for a domain?  If so, we could say if your
OS is non-resilient to preemption and your application requires
time monotonicity, you must specify gang scheduling.


> -----Original Message-----
> From: Keir Fraser [mailto:keir.fraser@xxxxxxxxxxxxx]
> Sent: Thursday, May 22, 2008 10:12 AM
> To: dan.magenheimer@xxxxxxxxxx; Xen-Devel (E-mail)
> Cc: Dave Winchell
> Subject: Re: [Xen-devel] [PATCH] [RFC] Building guests on 
> monotonic Xen
> system time
> 
> 
> Well, really it's baked into the design of some of the timer 
> modes that time
> can differ across VCPUs: it's their reason for existence. We 
> could say we
> don't support them any more -- I'm not sure anyone enables them for
> productised releases of Xen -- but we should at least do that 
> explicitly
> rather than subtly breaking them.
> 
> What I'll do is append doing some more work to your patch to 
> my work queue.
> I would say I'm moderately confident that your patch actually 
> will work fine
> for the timer modes that you guys are using in your packaging 
> of Xen, but I
> think some more architectural cleanup in this area would be good for
> xen-unstable. Whether that is remove some of the older timer 
> modes now, or
> clean up hvm_[sg]et_guest_time() to work nicely with those 
> modes, is open to
> debate.
> 
>  -- Keir
> 
> On 22/5/08 17:05, "Dan Magenheimer" 
> <dan.magenheimer@xxxxxxxxxx> wrote:
> 
> > Thanks for the review and the modifications!
> >
> > It does allow time to temporarily deviate across VCPUs, but
> > doesn't allow them to "apparently" deviate; that is a VCPU
> > can never observe or set a time which is older than the time
> > that another VCPU previously observed or set.
> >
> > Unless you can guarantee this, there's no way to avoid "Time
> > going backwards" errors, is there?
> >
> > Admittedly, I don't fully understand the inner workings of
> > all the timer modes, but if this doesn't fly for one or more
> > of the modes, doesn't this mean it is impossible for those
> > timer modes to guarantee monotonicity -- at least without
> > also requiring gang scheduling?
> >
> > Thanks,
> > Dan
> >
> > P.S. I had left the naming as hvm_get_guest_time_mono()
> > because I wasn't sure there would never be a version
> > needed that was non-mono, so thought it would be better
> > to be explicit in the name.  (I was especially sensitized
> > to this after having just completed teasing apart the
> > hvm_get_guest_tsc() from hvm_get_guest_time() ;-)
> >
> >> -----Original Message-----
> >> From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
> >> [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx]On Behalf Of 
> Keir Fraser
> >> Sent: Thursday, May 22, 2008 2:46 AM
> >> To: dan.magenheimer@xxxxxxxxxx; Xen-Devel (E-mail)
> >> Subject: Re: [Xen-devel] [PATCH] [RFC] Building guests on
> >> monotonic Xen
> >> system time
> >>
> >>
> >> Attached is a modified version (really just renames). 
> However the new
> >> hvm_set_guest_time() interface doesn't work for vpt.c timer
> >> modes which
> >> allow progress of time to temporarily deviate across VCPUs. I
> >> guess you
> >> don't use those modes so might not notice this issue. I think
> >> hvm guest time
> >> handling may need to be integrated into vpt.c so that correct
> >> handling can
> >> be applied for the various different types of timer mode.
> >> Having a blanket
> >> per-domain stime offset and enforcing monotonicity regardless
> >> of timer mode
> >> doesn't fly.
> >>
> >>  -- Keir
> >>
> >> On 21/5/08 20:01, "Dan Magenheimer"
> >> <dan.magenheimer@xxxxxxxxxx> wrote:
> >>
> >>> OK, I think this version is ready for prime-time.
> >>>
> >>> Keir, please check-in to unstable.
> >>>
> >>> [PATCH] Build hvm guest platform timers on monotonic Xen 
> system time
> >>>
> >>> This patch moves hvm platform timers from underlying
> >> physical CPU TSC
> >>> to Xen system time and ensures monotonicity.  TSC on many
> >> systems may
> >>> skew between processors, thus hvm platform timer reads were not
> >>> monotonic which led to "Time going backwards" problems.
> >>>
> >>> Signed-off-by: Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>
> >>>
> >>>> -----Original Message-----
> >>>> From: Keir Fraser [mailto:keir.fraser@xxxxxxxxxxxxx]
> >>>> Sent: Tuesday, May 20, 2008 1:37 AM
> >>>> To: dan.magenheimer@xxxxxxxxxx; Xen-Devel (E-mail)
> >>>> Subject: Re: [Xen-devel] [PATCH] [RFC] Building guests on
> >>>> monotonic Xen
> >>>> system time
> >>>>
> >>>>
> >>>> On 19/5/08 19:27, "Dan Magenheimer"
> >>>> <dan.magenheimer@xxxxxxxxxx> wrote:
> >>>>
> >>>>> Why? Scaling and adjusting of xen-time-based-tsc will
> >>>>> be very difficult to coordinate with processor-based-tsc.
> >>>>> We need to always ensure that A < B < C for a guest
> >>>>> executing:
> >>>>>
> >>>>> rdtsc(A) /* untrapped */
> >>>>> emulated_rdtsc(B)
> >>>>> rdtsc(C) /* untrapped */
> >>>>>
> >>>>> Further, OS's use TSC as a highest-resolution time source
> >>>>> with knowledge that TSCs on different processors may
> >>>>> not be synchronized, whereas they assume that a platform
> >>>>> timer is one-per-system and monotonically increasing.
> >>>>>
> >>>>> Keir, if you disagree and see guest-TSC-on-Xen-system-time
> >>>>> as an absolute must, please let me know.
> >>>>
> >>>> I am inclined to say we should have a
> >>>> guest-TSC-on-system-time mode where
> >>>> *all* RDTSC executions get trapped. This would at least be
> >> useful as a
> >>>> baseline for tracking down guest time problems, and also 
> provide a
> >>>> guaranteed stable TSC timesource for those who care about
> >>>> that more than
> >>>> pure performance.
> >>>>
> >>>> *However* I would agree that, with TSC virtualisation as it
> >>>> currently is,
> >>>> there actually isn't really a way to build guest TSC on Xen
> >>>> system time
> >>>> without periodically warping the TSC back or forth. The guest
> >>>> TSC runs at
> >>>> the host TSC rate and that is that!
> >>>>
> >>>> I think my original point was that at least we should not
> >>>> build all our
> >>>> other virtual time sources on this dodgy 'guest TSC'. :-)
> >>>>
> >>>>  -- Keir
> >>>>
> >>>>
> >>>>
> >>
> >>
> >>
> >
> 
> 
>


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel