[Xen-devel] RE: Guest TSC and Xen (Intel and AMD feedback please)

["Dan Magenheimer" <dan.magenheimer@xxxxxxxxxx>]

Hi Andy --

Thanks for the reply.

No, I don't think the TSC offset capabilities in VT are sufficient.
If you are migrating from a TSC-synchronized SMP system and TSC
was selected as the clocksource by the guest at boot *because* TSC
is always synchronized, and then you migrate to a system where TSC
is not synchronized, Xen can synchronize it once at migrate time
but then the TSC's on the target system will immediately start to
diverge.  So TSC might be a reasonable clocksource on the first
system but not on the target system.  One could of course use
CPUID to disallow a TSC-unsynchronized host as a suitable target
for a TSC-synchronized-assumed guest, but that seems overly restrictive,
especially if TSC wasn't selected as the clocksource for the guest
and/or the guest (and its apps) isn't particularly time-sensitive.

> I don't know why you want to single out TSC here.

I'm singleing it out because it is a per-cpu clock rather
than a platform timer... a platform timer can be (and indeed
is) offset'ed on migration and that is sufficient if it is
selected as the clocksource.

> That is what Linux is testing for anyways. If it decides it is
> ok it is fine.

Not sure... if Linux thinks it is running on a uniprocessor,
but Xen reschedules this uniprocessor Linux guest on a different
processor on the same physical SMP system, does Xen adjust the
potential TSC difference?  I could be wrong, but I think not.

> The reason why it is an advantage to try to make TSC btw
> is that it is *much* faster than any other timer and there
> are definitely workloads that are very timer intensive.

Yes, understood, but if a timer-intensive application makes
the assumption that TSC is synchronized and thus will never
go backwards, but TSC is not synchronized and it DOES (apparently)
go backwards due to Xen scheduler or migration, a slower timer
might have been preferred.

Dan

> -----Original Message-----
> From: Andi Kleen [mailto:andi@xxxxxxxxxxxxxx]
> Sent: Tuesday, July 01, 2008 2:42 PM
> To: dan.magenheimer@xxxxxxxxxx
> Cc: Xen-Devel (E-mail)
> Subject: Re: Guest TSC and Xen (Intel and AMD feedback please)
> 
> 
> "Dan Magenheimer" <dan.magenheimer@xxxxxxxxxx> writes:
> >
> > 1) Often this test is done once at guest boot; if the guest
> >    migrates to another machine without the bit set, time
> >    will be erratic.
> 
> First it doesn't have to be. Xen could use the TSC offset capabilities
> in VT to simulate a relatively [you would likely get a small hickup
> during the migration, but nothing really bad] stable TSC between
> machines per guest (assuming the TSC does not diverge by itself).
> That would require a TSC negotation phase during migration.
> 
> Then the tests are just a special instance of a CPUID bit test
> [although they sometimes check other things too, but from this
> perspective they are the same] and when your CPUID and other
> configuration values are not controlled between different systems you
> migrate to you cannot safely migrate anyways because everything else
> that depends on CPUID features will fail too.
> 
> I don't know why you want to single out TSC here.
> 
> > 2) I *think* that in some cases even within the same system,
> >    TSC values will skew somewhat.  Since a uniprocessor guest
> >    will often be rescheduled to a different pcpu by Xen,
> >    the underlying tsc may appear erratic.
> 
> That is what Linux is testing for anyways. If it decides it is
> ok it is fine.
> 
> The reason why it is an advantage to try to make TSC btw
> is that it is *much* faster than any other timer and there
> are definitely workloads that are very timer intensive.
> 
> -Andi
>


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel