RE: [Xen-devel] [BUG 1282] time jump on live migrate root cause & proposed fixes

["Dan Magenheimer" <dan.magenheimer@xxxxxxxxxx>]

An idea I've been considering to handle live migration between
"good tsc" and "bad tsc" machines may work nicely here too:

1) Implement softtsc* on PV domains (per-domain rather than global)
2) In live migrate startup, force-switch domain to scaled softtsc
3) When control transfers to new machine, rescale softtsc to tsc
   rate on target hypervisor
4) In live migrate cleanup, switch domain off of softtsc (or
   not, if Xen determines the tsc's on target are untrustworthy)

I'd also like to see softtsc controllable by a sysfs.

Dan

* softtsc means trap all tsc reads/writes and emulate them in
  the hypervisor.  This works (in 4.0) for hvm domains and tsc
  reads on my machine go from ~80ns to ~3us... not great but
  not horrible.

> -----Original Message-----
> From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
> [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx]On Behalf Of 
> Rik van Riel
> Sent: Wednesday, August 06, 2008 2:47 PM
> To: xen-devel@xxxxxxxxxxxxxxxxxxx
> Subject: [Xen-devel] [BUG 1282] time jump on live migrate root cause &
> proposed fixes
> 
> 
> Hi,
> 
> I have done some debugging to find out the root cause of bug 
> 1282, which
> has the following symptoms with paravirtualized guests:
> - after a live migrate, the time on the guest can jump
> - after a live migrate, the guest "forgets" to wake up processes
> - after a domU save, dom0 reboot and domU restore, the time is
>   correct but processes are not woken up from sys_nanosleep
> 
> The problem seems to stem from the fact that domU uses the 
> hypervisor's
> system_time, which is the time since hypervisor system bootup in
> nanoseconds, as its base for timekeeping.
> 
> This works fine as long as the guest stays on the same hypervisor,
> but if the guest is migrated to a hypervisor with a different uptime,
> problems ensue.  Specifically, if the guest is migrated to a host
> with a lower uptime, processes that call sys_nanosleep() will not
> be woken up until the new host's uptime catches up with the uptime
> of the old host!   While waiting for the uptime to catch up,
> gettimeofday always returns the same value.
> 
> Conversely, if a guest migrates from a host with a lower uptime to
> a host with a higher uptime, the system time in the guest advances
> by the difference between the two uptimes.
> 
> 
> I can think of a few possible fixes for this issue:
> 
> 1) have system_time in the hypervisor start at unix epoch 0
>    (january 1st 1970) instead of at boot time - this may
>    require some magic to sync_cmos_clock(), sync_xen_wallclock()
>    and/or other functions so dom0 does not get too confused while
>    changing the time during bootup
> 
> 2) have time_init() and time_resume() calculate the hypervisor
>    boot time from the shared_info ->wc_sec ->wc_nsec and the
>    shared_info->per cpu vcpu_info->system_time -- if the host
>    boot time changes (by more than a second?) adjust some local
>    offset that we add into get_nsec_offset() and get_usec_offset()
>    to always adjust the time right
> 
> 3) get_time_values_from_xen() and __update_wallclock() can keep
>    track of such an offset by themselves
> 
> 
> Does anybody have comments on the ideas above, or maybe even
> better ideas on how to fix the problem? :)
> 
> --
> All Rights Reversed
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel
>


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel