[PATCH] softtsc (was RE: [Xen-devel] Guest TSC and Xen (Intel and AMD feedback please))

["Dan Magenheimer" <dan.magenheimer@xxxxxxxxxx>]

The attached patch adds a new Xen boot option "softtsc" that
causes all hvm domains to trap and emulate all rdtsc instructions
and fetch Xen system time instead of the hardware tsc.  As
a result, the processor will appear to be running at 1000.xyz MHz
(where xyz indicates some fraction that seems to vary slightly
across different boots).  I've also added a line of output to
"xm debug-key t" to report softsc and tsc-related CPU features.

(Keir, though 3.3 functionality freeze is past, this is more of a
diagnostic aid and defaults off, so I don't know if you would
consider it for 3.3.)

Signed-off-by: Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>

Dan

P.S. Thanks to Li Zhang for coding the foundation for this patch.
Li, do you want to provide a signed-off-by line too?

> -----Original Message-----
> From: Dan Magenheimer [mailto:dan.magenheimer@xxxxxxxxxx]
> Sent: Tuesday, July 08, 2008 6:30 PM
> To: 'Zhang, Li'; 'Xen-Devel (E-mail)'
> Cc: 'Tian, Kevin'
> Subject: RE: [Xen-devel] Guest TSC and Xen (Intel and AMD 
> feedback please)
> 
> 
> Looks great!  I am continuing to code and test.  Some 
> preliminary results:
> 
> 1) Substituting hvm_get_guest_time() for hvm_get_guest_tsc() 
> seems to just work!  The hvm
>     guest believes it is running on a 1000.185 MHz clock.
> 
> 2) On my test box, a guest rdtsc without softtsc takes about 
> 30ns.  With softtsc,
>     rdtsc takes about 1us.
> 
> 3) On my user-land test program, I am seeing rare "Time goes 
> backwards", even with
>    hvm_get_guest_time().  I'm trying to track this down.
> 
> Dan
> 
> > -----Original Message-----
> > From: Zhang, Li [mailto:li.zhang@xxxxxxxxx]
> > Sent: Tuesday, July 08, 2008 3:47 AM
> > To: Zhang, Li; dan.magenheimer@xxxxxxxxxx; Xen-Devel (E-mail)
> > Cc: Tian, Kevin
> > Subject: RE: [Xen-devel] Guest TSC and Xen (Intel and AMD 
> > feedback please)
> > 
> > 
> > Hi, Dan
> > Please ignore previous comments. The above attachment is OK. 
> > Sorry for that. :(
> > 
> > >  -----Original Message-----
> > >  From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
> > >  [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf 
> > Of Zhang, Li
> > >  Sent: Tuesday, July 08, 2008 2:58 PM
> > >  To: dan.magenheimer@xxxxxxxxxx; Xen-Devel (E-mail)
> > >  Cc: Tian, Kevin
> > >  Subject: RE: [Xen-devel] Guest TSC and Xen (Intel and AMD 
> > feedback please)
> > >
> > >  In fact, the issue is from guest. In the guest's terminal, 
> > it shows that "your
> > >  time source seems to be instable..."
> > >
> > >  >  -----Original Message-----
> > >  >  From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
> > >  >  [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On 
> > Behalf Of Zhang, Li
> > >  >  Sent: Tuesday, July 08, 2008 2:49 PM
> > >  >  To: dan.magenheimer@xxxxxxxxxx; Xen-Devel (E-mail)
> > >  >  Cc: Tian, Kevin
> > >  >  Subject: RE: [Xen-devel] Guest TSC and Xen (Intel and 
> > AMD feedback please)
> > >  >
> > >  >  Hi, Dan
> > >  >
> > >  >  This is the patch which I think it has been completed 
> > VT. But there is an
> > >  >  instable issue. The reason may be from that this 
> > interception is not bound
> > >  >  to timer mode yet.
> > >  >
> > >  >  Thanks
> > >  >  --Li
> > >  >
> > >  >  >  -----Original Message-----
> > >  >  >  From: Dan Magenheimer [mailto:dan.magenheimer@xxxxxxxxxx]
> > >  >  >  Sent: Tuesday, July 08, 2008 11:34 AM
> > >  >  >  To: Zhang, Li; Xen-Devel (E-mail)
> > >  >  >  Cc: Tian, Kevin; dan.magenheimer@xxxxxxxxxx
> > >  >  >  Subject: RE: [Xen-devel] Guest TSC and Xen (Intel 
> > and AMD feedback
> > >  please)
> > >  >  >
> > >  >  >  Thanks Kevin and Li --
> > >  >  >
> > >  >  >  A couple of questions:
> > >  >  >
> > >  >  >  1) If the EXITING flag to be set in vmcs.c is to be 
> > controlled by a xen
> > >  >  boot
> > >  >  >  option, would this work:
> > >  >  >
> > >  >  >  static int opt_softtsc = 0;
> > >  >  >  boolean_param("softtsc", opt_softtsc);
> > >  >  >    :
> > >  >  >  min = /* original code */
> > >  >  >  if (opt_softtsc) min |= CPU_BASED_RDTSC_EXITING;
> > >  >  >
> > >  >  >  2) In vmx_rdtsc_intercept(/* need regs as param*/) 
> > if we change the
> > >  >  >  rdtscll(tsc) to be tmptsc = hvm_get_guest_tsc() and 
> > then set regs->edx
> > >  amd
> > >  >  >  regs->eax from tmptsc, are we almost done?
> > >  >  >
> > >  >  >  3) Des Linux or Windows use the CR4.TSD flag and, if 
> > so, what code gets
> > >  >  called
> > >  >  >  to force the trap?
> > >  >  >
> > >  >  >  4) Does Linux or Windows use RDMSR/WRMSR of tsc?  Is 
> > there already code
> > >  >  >  somewhere to emulate WRMSR of tsc?
> > >  >  >
> > >  >  >  Sorry I am not very familar with the details of the 
> > ia32 instruction
> > >  set.
> > >  >  >
> > >  >  >  Thanks,
> > >  >  >  Dan
> > >  >  >
> > >  >  >  > -----Original Message-----
> > >  >  >  > From: Zhang, Li [mailto:li.zhang@xxxxxxxxx]
> > >  >  >  > Sent: Monday, July 07, 2008 9:00 PM
> > >  >  >  > To: dan.magenheimer@xxxxxxxxxx; Xen-Devel (E-mail)
> > >  >  >  > Cc: Tian, Kevin
> > >  >  >  > Subject: RE: [Xen-devel] Guest TSC and Xen (Intel and AMD
> > >  >  >  > feedback please)
> > >  >  >  >
> > >  >  >  >
> > >  >  >  > Hi, Dan
> > >  >  >  > This seems not to be enough. I will cook another patch.
> > >  >  >  >
> > >  >  >  > >  -----Original Message-----
> > >  >  >  > >  From: Zhang, Li
> > >  >  >  > >  Sent: Tuesday, July 08, 2008 10:28 AM
> > >  >  >  > >  To: Tian, Kevin; 'dan.magenheimer@xxxxxxxxxx'; 
> > 'Xen-Devel
> > >  (E-mail)'
> > >  >  >  > >  Subject: RE: [Xen-devel] Guest TSC and Xen 
> > (Intel and AMD
> > >  >  >  > feedback please)
> > >  >  >  > >
> > >  >  >  > >  Hi, Dan
> > >  >  >  > >
> > >  >  >  > >  I just add some code to produce vmexit. The function
> > >  >  >  > vmx_rdtsc_intercept()
> > >  >  >  > >  is not completed. It is needed to add some code 
> > to get the
> > >  >  >  > guest tsc.
> > >  >  >  > >
> > >  >  >  > >  Thanks
> > >  >  >  > >  --Li
> > >  >  >  > >
> > >  >  >  > >  >  -----Original Message-----
> > >  >  >  > >  >  From: Tian, Kevin
> > >  >  >  > >  >  Sent: Tuesday, July 08, 2008 9:39 AM
> > >  >  >  > >  >  To: 'dan.magenheimer@xxxxxxxxxx'; 
> Xen-Devel (E-mail)
> > >  >  >  > >  >  Cc: Zhang, Li
> > >  >  >  > >  >  Subject: RE: [Xen-devel] Guest TSC and Xen 
> (Intel and
> > >  >  >  > AMD feedback please)
> > >  >  >  > >  >
> > >  >  >  > >  >  >From: Dan Magenheimer 
> > [mailto:dan.magenheimer@xxxxxxxxxx]
> > >  >  >  > >  >  >Sent: 2008年7月5日 1:32
> > >  >  >  > >  >  >
> > >  >  >  > >  >  >> >Is this something that you (or Intel 
> in general)
> > >  >  >  > could look at?
> > >  >  >  > >  >  >> >I would be happy to participate but I 
> > don't think I
> > >  >  >  > understand
> > >  >  >  > >  >  >> >VT well enough.  Once the trap occurs, 
> I suppose
> > >  >  >  > Xen system time
> > >  >  >  > >  >  >> >could be used as the virtual TSC, 
> > possibly scaled up.
> > >  >  >  > >  >  >> >
> > >  >  >  > >  >  >>
> > >  >  >  > >  >  >> There should be tiny related to VT, as 
> > only turning
> > >  >  >  > on some bit to
> > >  >  >  > >  >  >> allow RDTSC trapping and then the rest 
> > stuff should be common
> > >  >  >  > >  >  >> how to handle it. We'll take a look, but can't
> > >  >  >  > commit the time due
> > >  >  >  > >  >  >> to other scheduled bandwidth. But if 
> you'd like to
> > >  >  >  > jump in early
> > >  >  >  > >  >  >> we definitely can help with VT side.
> > >  >  >  > >  >  >
> > >  >  >  > >  >  >If you can post a patch with code that:
> > >  >  >  > >  >  >
> > >  >  >  > >  >  >1) declares a boolean global variable: softtsc = 0
> > >  >  >  > >  >  >2) if the variable is set, a rdtsc 
> > instruction in any hvm
> > >  >  >  > >  >  >   domain causes a trap
> > >  >  >  > >  >  >3) the trap handler just does a physical 
> > rdtsc and returns
> > >  >  >  > >  >  >
> > >  >  >  > >  >  >then I could probably take it from there.
> > >  >  >  > >  >
> > >  >  >  > >  >  OK, and Li in CC will take a look and bake a 
> > patch for you.
> > >  >  >  > >  >
> > >  >  >  > >  >  Thanks,
> > >  >  >  > >  >  Kevin
> > >  >  >  >
> > >
> > >  _______________________________________________
> > >  Xen-devel mailing list
> > >  Xen-devel@xxxxxxxxxxxxxxxxxxx
> > >  http://lists.xensource.com/xen-devel
> > 
> >

Attachment: softtsc.patch
Description: Binary data

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel