Re: [Xen-devel] [PATCH] [RFC] Building guests on monotonic Xen system time

[Keir Fraser <keir.fraser@xxxxxxxxxxxxx>]

Well, really it's baked into the design of some of the timer modes that time
can differ across VCPUs: it's their reason for existence. We could say we
don't support them any more -- I'm not sure anyone enables them for
productised releases of Xen -- but we should at least do that explicitly
rather than subtly breaking them.

What I'll do is append doing some more work to your patch to my work queue.
I would say I'm moderately confident that your patch actually will work fine
for the timer modes that you guys are using in your packaging of Xen, but I
think some more architectural cleanup in this area would be good for
xen-unstable. Whether that is remove some of the older timer modes now, or
clean up hvm_[sg]et_guest_time() to work nicely with those modes, is open to
debate.

 -- Keir

On 22/5/08 17:05, "Dan Magenheimer" <dan.magenheimer@xxxxxxxxxx> wrote:

> Thanks for the review and the modifications!
> 
> It does allow time to temporarily deviate across VCPUs, but
> doesn't allow them to "apparently" deviate; that is a VCPU
> can never observe or set a time which is older than the time
> that another VCPU previously observed or set.
> 
> Unless you can guarantee this, there's no way to avoid "Time
> going backwards" errors, is there?
> 
> Admittedly, I don't fully understand the inner workings of
> all the timer modes, but if this doesn't fly for one or more
> of the modes, doesn't this mean it is impossible for those
> timer modes to guarantee monotonicity -- at least without
> also requiring gang scheduling?
> 
> Thanks,
> Dan
> 
> P.S. I had left the naming as hvm_get_guest_time_mono()
> because I wasn't sure there would never be a version
> needed that was non-mono, so thought it would be better
> to be explicit in the name.  (I was especially sensitized
> to this after having just completed teasing apart the
> hvm_get_guest_tsc() from hvm_get_guest_time() ;-)
> 
>> -----Original Message-----
>> From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
>> [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx]On Behalf Of Keir Fraser
>> Sent: Thursday, May 22, 2008 2:46 AM
>> To: dan.magenheimer@xxxxxxxxxx; Xen-Devel (E-mail)
>> Subject: Re: [Xen-devel] [PATCH] [RFC] Building guests on
>> monotonic Xen
>> system time
>> 
>> 
>> Attached is a modified version (really just renames). However the new
>> hvm_set_guest_time() interface doesn't work for vpt.c timer
>> modes which
>> allow progress of time to temporarily deviate across VCPUs. I
>> guess you
>> don't use those modes so might not notice this issue. I think
>> hvm guest time
>> handling may need to be integrated into vpt.c so that correct
>> handling can
>> be applied for the various different types of timer mode.
>> Having a blanket
>> per-domain stime offset and enforcing monotonicity regardless
>> of timer mode
>> doesn't fly.
>> 
>>  -- Keir
>> 
>> On 21/5/08 20:01, "Dan Magenheimer"
>> <dan.magenheimer@xxxxxxxxxx> wrote:
>> 
>>> OK, I think this version is ready for prime-time.
>>> 
>>> Keir, please check-in to unstable.
>>> 
>>> [PATCH] Build hvm guest platform timers on monotonic Xen system time
>>> 
>>> This patch moves hvm platform timers from underlying
>> physical CPU TSC
>>> to Xen system time and ensures monotonicity.  TSC on many
>> systems may
>>> skew between processors, thus hvm platform timer reads were not
>>> monotonic which led to "Time going backwards" problems.
>>> 
>>> Signed-off-by: Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>
>>> 
>>>> -----Original Message-----
>>>> From: Keir Fraser [mailto:keir.fraser@xxxxxxxxxxxxx]
>>>> Sent: Tuesday, May 20, 2008 1:37 AM
>>>> To: dan.magenheimer@xxxxxxxxxx; Xen-Devel (E-mail)
>>>> Subject: Re: [Xen-devel] [PATCH] [RFC] Building guests on
>>>> monotonic Xen
>>>> system time
>>>> 
>>>> 
>>>> On 19/5/08 19:27, "Dan Magenheimer"
>>>> <dan.magenheimer@xxxxxxxxxx> wrote:
>>>> 
>>>>> Why? Scaling and adjusting of xen-time-based-tsc will
>>>>> be very difficult to coordinate with processor-based-tsc.
>>>>> We need to always ensure that A < B < C for a guest
>>>>> executing:
>>>>> 
>>>>> rdtsc(A) /* untrapped */
>>>>> emulated_rdtsc(B)
>>>>> rdtsc(C) /* untrapped */
>>>>> 
>>>>> Further, OS's use TSC as a highest-resolution time source
>>>>> with knowledge that TSCs on different processors may
>>>>> not be synchronized, whereas they assume that a platform
>>>>> timer is one-per-system and monotonically increasing.
>>>>> 
>>>>> Keir, if you disagree and see guest-TSC-on-Xen-system-time
>>>>> as an absolute must, please let me know.
>>>> 
>>>> I am inclined to say we should have a
>>>> guest-TSC-on-system-time mode where
>>>> *all* RDTSC executions get trapped. This would at least be
>> useful as a
>>>> baseline for tracking down guest time problems, and also provide a
>>>> guaranteed stable TSC timesource for those who care about
>>>> that more than
>>>> pure performance.
>>>> 
>>>> *However* I would agree that, with TSC virtualisation as it
>>>> currently is,
>>>> there actually isn't really a way to build guest TSC on Xen
>>>> system time
>>>> without periodically warping the TSC back or forth. The guest
>>>> TSC runs at
>>>> the host TSC rate and that is that!
>>>> 
>>>> I think my original point was that at least we should not
>>>> build all our
>>>> other virtual time sources on this dodgy 'guest TSC'. :-)
>>>> 
>>>>  -- Keir
>>>> 
>>>> 
>>>> 
>> 
>> 
>> 
> 



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel