[Xen-devel] [RFC] Correct/fast timestamping in apps under Xen [0 of 4]

[Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>]

Over the last few weeks and in a number of xen-devel
threads, I've been trying to determine a solution
that would allow apps running on Xen to obtain timestamp
information (i.e. rdtsc) in a way which is both correct
and "as fast as possible" in a wide range of hardware
and software conditions.  This is important because:
(a) some apps need to obtain a timestamp up to a hundred
thousand times/second or more; (b) direct ("native")
use of the rdtsc instruction in a Xen environment
can lead to extremely subtle bugs with potentially
extreme consequences up to and including data corruption
("correctness" issues); and (c) alternate approaches
for obtaining timestamp information are too
slow (emulation, rdtsc exiting, OS intrinsics).

Thanks very much to the ideas and guidance from
many of you, I think I have a reasonable plan for
a design and implementation.  However, it is very
dependent on a few premises which I will summarize
briefly here and then expand on below.  Since
the premises introduce both new concepts and
philosophies for Xen and some dependencies
on processor/server capabilities not widely known
or understood (or, for historical reasons, trusted),
and since a complete solution depends on ALL
the premises, I wanted to get review and comment
on ALL of them before commencing implementation
of any.  I suspect the implementation will take
less time than the discussion generated ;-)
Further, Keir has stated that he doesn't want
isolated patches working in this general direction
until he sees the big picture.

Since the premises are diverse, and since it's
easy to lose interest in a string of responses unrelated
to one expert's area of interest, I will followup
with a separate thread for each, but let me
briefly summarize them here first.

The premises are:

1) A large and growing percentage of servers running
   Xen have a "reliable" TSC and Xen can determine
   conclusively whether a server does or does not
   have a reliable TSC.
2) A small but growing percentage of servers running
   Xen implement the rdtscp instruction but Xen does
   not and will not expose this instruction to guest
   OSes.
3) Xen is able to track the "incarnation" number for
   a guest.  This number will increase whenever a
   guest is restored or migrated (and possibly more
   frequently).  Optionally, an administrator can
   explicitly mark a guest as "landlocked", disallowing
   save/restore/migration for that guest.
4) Apps can become "virtualization aware" in that
   they can access certain information directly
   from Xen utilizing an OS-independent mechanism.
   This information includes not only "Am I running
   on Xen?" but also, for example, "Is TSC reliable
   on this physical machine?", "Is rdtsc emulated
   or native on this virtual machine?",  "What is
   the current incarnation number for this virtual
   machine?", "Is this virtual machine landlocked?",
   "What are the pvclock parameters for this
   virtual machine?", etc.

I'm not trying to be coy... it's probably not
difficult to deduce the proposal from the above
and previous posts.  I just want to focus first
on the validity of the premises.

If you have comment on any specific premise, please
reply to the subsequent [x of 4] message.  If
you have comment on the whole direction/philosopy
(though I hope we've already beaten that to death :-),
please reply to this [0 of 4] message.

Thanks,
Dan

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel