Re: [MirageOS-devel] NTP client integration with MirageOS / Mini-OS

[Thomas Leonard <talex5@xxxxxxxxx>]

On 20 July 2016 at 18:10, Hannes Mehnert <hannes@xxxxxxxxxxx> wrote:
> Hi Kia,
>
> On 20/07/2016 18:54, Kia wrote:
>> I have a working NTP client written in ocaml and that runs in mirageos. I 
>> still
>> need to do cleanup work, write and verify code that properly handles leap
>> second events, write .mli files, and write tests -- but the client code 
>> works.
>> I have a unikernel which periodically queries an NTP server to generate data
>> that lets other code in the unikernel convert the current value of RDTSC [0] 
>> to
>> an absolute time:
>>
>> https://github.com/matildah/mirage-ntp/blob/a5d06d05cc20bb9af680238437aeea5835fc54fa/unikernel/unikernel.ml#L58-L63
>>
>> The algorithms used to convert timestamped (with RDTSC when they are
>> sent/received) NTP packets to offset/rate information that can be used to
>> convert the current RDTSC value to a time (or a pair of counter values to a
>> difference of time) are a reimplementation of RADclock's (source at
>> https://github.com/synclab/radclock), papers at 
>> http://www.synclab.org/docs/).
>> RADclock has been extensively tested with months of data (and shown to give
>> accuracy an order of magnitude better than the reference NTP implementation 
>> in
>> the same conditions) -- http://www.synclab.org/testbed/ has details and my 
>> post
>> https://matildah.github.io/posts/2016-05-23-ntp-status.html concisely 
>> explains
>> the advantages of RADclock's feed-forward clock synchronization and its
>> suitability for a unikernel environment.
>>
>>
>> The user-facing parts of the client code are:
>> A type representing the current state of the NTP client, "state":
>> https://github.com/matildah/mirage-ntp/blob/a5d06d05cc20bb9af680238437aeea5835fc54fa/lib/client/types.ml#L196-L202
>> A function that generates a new NTP query, "new_query":
>> https://github.com/matildah/mirage-ntp/blob/a5d06d05cc20bb9af680238437aeea5835fc54fa/lib/client/ntp_client.ml#L66-L69
>> A function that updates the current state with the NTP server's reply, 
>> "add_sample":
>> https://github.com/matildah/mirage-ntp/blob/a5d06d05cc20bb9af680238437aeea5835fc54fa/lib/client/ntp_client.ml#L118-L121
>> A type that contains current rate/offset estimates, "output":
>> https://github.com/matildah/mirage-ntp/blob/a5d06d05cc20bb9af680238437aeea5835fc54fa/lib/client/types.ml#L73-L80
>> A function that takes the current NTP client's state and extracts the 
>> current rate/offset estimates, "output_of_state":
>> https://github.com/matildah/mirage-ntp/blob/a5d06d05cc20bb9af680238437aeea5835fc54fa/lib/client/ntp_client.ml#L123-L136
>>
>> and you can see them called in the demonstration unikernel code at
>> https://github.com/matildah/mirage-ntp/blob/a5d06d05cc20bb9af680238437aeea5835fc54fa/unikernel/unikernel.ml#L51-L63
>>
>> The client code can take the value returned by output_of_state and the 
>> current value of RDTSC
>> and use 
>> https://github.com/matildah/mirage-ntp/blob/a5d06d05cc20bb9af680238437aeea5835fc54fa/lib/clocklib/diffabs.ml#L3-L10
>> to generate an absolute time. (I haven't written the difference time 
>> function yet).
>
> This is great!
>
>> I have a few questions about how to properly integrate my client code into a
>> module that can be used like https://github.com/mirage/mirage-clock by any
>> unikernel that needs the time (but doesn't depend on time information from
>> dom0).
>>
>> 1. How can I make my NTP client module depend on the mirage RANDOM module and
>> call int32 from it? (The new_query function needs to be able to get some
>> randomness to put in the packet as a nonce)
>
> There are two options here
>  - if it is code already depending on mirage-types, define a functor (as
> done e.g. in UDP
> https://github.com/mirage/mirage-tcpip/blob/ed6dd67f7ed47acba49c8021546a4b9c38d3359c/lib/udp/udp.ml).
>  this will then use the MirageOS Random device.
>  - if the code otherwise does not depend on mirage, depending on which
> random you need (real or fake), use Nocrypto.Rng.generate 4 (and depend
> in opam on nocrypto), or Random.int32
>
>> 2. Is there way to have a module that starts a background thread (in my case,
>> that queries the NTP server, updates its estimates of time counter 
>> rate/offset,
>> and then sleeps until the next poll interval) but also enable other threads 
>> in
>> the unikernel to get the current value of a variable in its context?

The CLOCK signature uses

  val time: unit -> float

rather than

  val time: t -> float

for some reason, which makes this difficult. I suggest changing the
signature to work like the other devices.

You could use a global variable to hold the current time, but that's
quite messy.

Starting a background thread is easy enough, but stopping it is
trickier. Perhaps there should be `listen` and `disconnect` functions
on CLOCK too. For the current clocks these aren't needed, but for an
NTP clock it would be useful.

> There is atm no such way AFAIK.  In the mirage tool there is hardcoded
> some code to start entropy harvesting, and there is the mirage-logs
> which is started "magically" (but wasn't the mirage-logs integration
> done in a more general way)?

Logging had to be magical because:

1. We need to start it before everything else, except things it depends on.
2. Things use logging without explicitly depending on it.
3. We wanted logging to be enabled by default even in unikernels that
don't explicitly request logging.

I don't think these apply to time. Components that want time should
ask for it explicitly.

>> Failing that, what way can I have a background thread that continually runs 
>> and
>> always makes available the latest calculated value of "output" to other 
>> threads
>> (so they can compute the current time based on the current RDTSC value)?
>>
>> Morally, my NTP client thread needs to let other threads have access to its
>> latest rate/offset information so they can calculate the current time (much 
>> how
>> Xen makes available vcpu_time_info in a shared page), but I do not know how
>> to set this up in mirage.
>
> At the moment, mirage-clock-unix/xen provide the CLOCK (soon PCLOCK)
> interface.  The NTP client should implement the same interface, and as a
> user I'd like to use this and pass it on to other CLOCK consumers.
>
> This reminds me of the hardcoded tcpip.arpv4 stuff inside the mirage
> tool.  Maybe we should re-think the code stubs in there now that we have
> multiple implementation of core interfaces?  Maybe stackv4 etc. should
> receive optional parameters for arp/clock/... (a brief look into
> lib/mirage.ml shows that they already do!?! (maybe I just have to try
> this out))?


-- 
talex5 (GitHub/Twitter)        http://roscidus.com/blog/
GPG: 5DD5 8D70 899C 454A 966D  6A51 7513 3C8F 94F6 E0CC
GPG: DA98 25AE CAD0 8975 7CDA  BD8E 0713 3F96 CA74 D8BA

_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel