Re: [MirageOS-devel] vchan hackers wanted for mirage-entropy

[Nicolas Ojeda Bar <no263@xxxxxxxxxxxxxxx>]

Hi list,

After discussion, it seems that the vchan/xenstore libraries are not
up to where they should be to make this work.  In the very short term,
it was suggested to generate a seed from /dev/urandom at compile time
and then use that to seed the entropy generator.

Yesterday night I pushed PRs to `mirage`
(https://github.com/mirage/mirage/pull/323) and `mirage-entropy`
(https://github.com/mirage/mirage-entropy/pull/7) to implement this
idea.
It is hacky (esp. the modifications to the configure script), but it
should work.  Any comments welcome.

Cheers,
Nicolas

On Thu, Nov 20, 2014 at 7:29 PM, Nicolas Ojeda Bar
<no263@xxxxxxxxxxxxxxx> wrote:
> FYI
>
> ---------- Forwarded message ----------
> From: Dave Scott <Dave.Scott@xxxxxxxxxx>
> Date: Thu, Nov 20, 2014 at 2:55 PM
> Subject: Re: ocaml-xenstore questions
> To: Nicolas Ojeda Bar <no263@xxxxxxxxxxxxxxx>
>
>
> Hi,
>
>> On 20 Nov 2014, at 14:29, Nicolas Ojeda Bar <no263@xxxxxxxxxxxxxxx> wrote:
>>
>> Dear Dave,
>>
>> I have some questions and Anil suggested I emailed you about them.
>>
>> A little explanation:
>>
>> I am trying to use ocaml-xenstore to build a server in dom0 that will
>> serve entropy from /dev/urandom to any unikernel that wants it via
>> vchans.  I can already send the data using ocaml-vchan (manually
>> setting the domid of the client) and now I want to use the xenstore to
>> negotiate the client connections so that we do not need to enter the
>> domid manually.  There is some code in conduit that seems to be what
>> we need (Mirage_xenstore).
>
> I think the conduit negotiation code works well enough as a prototype,
> but itâs not secure enough for proper use because of the limitations
> of the xenstore access control system.
>
> In a Xen system the âtoolstackâ (xend, xl, xapi etc) uses its
> privileged access to create backend and frontend directories, setting
> the permissions and introducing the domains to each other. In the
> conduit prototype we made it peer to peer (no toolstack) but this
> meant we had to grant the equivalent to âchmod a+rwxâ to a xenstore
> treeâ this means that anyone can interfere with anyone elseâs
> connections.
>
> https://github.com/mirage/ocaml-vchan/blob/master/lib_test/mirage/init-xenstore.sh#L6
>
> The xenstore protocol (of which the access control stuff is part)
> evolves very slowly because itâs a core part of the system. I think we
> should explore changing that but it will take a while. In the meantime
> I think you could solve the problem in a more (Xen-)traditional
> fashion by creating a privileged daemon which runs in dom0 and acts
> like the toolstack. It would
>
> * watch for the arrival of new domains via the @introduceDomain watch
> * create a vchan connection to each one directly
>
>>
>> Questions:
>>
>> - The latest ocaml-xenstore in opam is 1.2.5.  It seems to be quite
>> out of date.  I pinned the current github repo but then mirage-xen
>> stopped compiling.  Is this supposed to happen ? Do you know how to
>> set things up to use the ocaml-xenstore trunk ?
>
> The trunk contains interface changes which are still being developed.
> When the interface changes are finished Iâll go around and update all
> the clients (block driver, network driver, console driver etc). I
> recommend avoiding this and sticking with 1.2.5 :-)
>
>
>> - Mirage_xenstore in ocaml-conduit uses the OS.Xs module (a Xenstore
>> client).  This module will not work in dom0 because the Unix backend
>> of OS does not have a Xs submodule.  Anil thought that there was a
>> unix implementation of a Xenstore client that could be used instead.
>> Do you happen to know where to look for this ?
>
> Have a look at the mirage console code:
>
> https://github.com/mirage/mirage-console/blob/master/cli/main.ml
>
> In particular this is a CLI which runs in dom0 and makes console
> (rather than vchan) connections to other domains.
>
> Watching xenstore for new domains is such a common requirement that
> Iâd like to extend the xenstore API to include it. For now thereâs
> some code here which can do it:
>
> https://github.com/xapi-project/xenops/blob/master/src/xenstore_watch.ml#L48
>
>
>> I believe that you will be by the Computer Lab later today, so I may
>> pester you with more questions.
>>
>> Thank you very much!
>
> Itâs also worth having a look at the qemu virtio-rng driver. Itâs got
> some logic in there for rate-limiting â it would be good to understand
> whether we need that too.
>
> Good luck! :-)
>
> Cheers,
> Dave
>
>
>>
>> Best wishes,
>> Nicolas
>
> On Thu, Nov 20, 2014 at 11:36 AM, Anil Madhavapeddy <anil@xxxxxxxxxx> wrote:
>> On 20 Nov 2014, at 09:57, Nick Hardiman <nick@xxxxxxxxxxxxxxxxxxxxxx> wrote:
>>
>>
>> On 20 Nov 2014, at 05:02, david <unitedbiscuits@xxxxxxxxx> wrote:
>>
>>
>>  i think it would be nice to assemble a blog post or set of instructions or
>> something that describe the various ways of running xen in a development
>> environment for mirage. (and deployment environments for that matter.)
>>
>>
>> Speaking as an amateur idiot, I'd say there are missing steps for a Xen
>> install that should be come before http://openmirage.org/wiki/install. I
>> look forward to following the instructions.
>>
>>
>> Agreed. It's become easier in recent years with the Ubuntu packaging
>> improving, but it's by no means well documented.
>>
>> I just rebooted my VMWare Fusion Ubuntu setup with Xen and now the virtual
>> graphics driver bombs out and hangs for some reason.  Ho hum...over to
>> Virtualbox for me too I guess.
>>
>> Magnus: if you kick off the blog post about your setup, we can cut-and-paste
>> that into a more centralised version to benefit other folk as well.  Nik's
>> excellent work on the Vagrant setup should also be of use -- has anyone
>> tried it? (it's at https://github.com/mirage/mirage-vagrant-vms)
>>
>> -anil
>>
>>
>> _______________________________________________
>> MirageOS-devel mailing list
>> MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
>> http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel
>>
>
> On Wed, Nov 19, 2014 at 5:24 PM, Nicolas Ojeda Bar
> <no263@xxxxxxxxxxxxxxx> wrote:
>> Yep, I got one so I should be set.
>>
>> Cheers,
>> Nicolas
>>
>> On Wednesday, November 19, 2014, Anil Madhavapeddy <anil@xxxxxxxxxx> wrote:
>>>
>>> Great!  One minor annoyance with vchan is that it does require a Xen
>>> machine to establish communications. If you don't have a Xen host, a
>>> Cubieboard is the easiest way to get started in the short term, or a
>>> Virtualbox setup.  I believe that Magnus is writing down the instructions
>>> for his Virtualbox setup at the moment...
>>>
>>> -anil
>>>
>>> On 19 Nov 2014, at 17:19, Nicolas Ojeda Bar <no263@xxxxxxxxxxxxxxx> wrote:
>>>
>>> Hi Anil,
>>>
>>> I can try my hand at this; I need a break and wouldn't mind doing this
>>> while I write the Ipv6 blog post.
>>>
>>> Cheers,
>>> Nicolas
>>>
>>> On Wednesday, November 19, 2014, Anil Madhavapeddy <anil@xxxxxxxxxx>
>>> wrote:
>>>>
>>>> Anyone got time for this?  Writing the dom0 proxy is pretty much a
>>>> hello-world use of the vchan bindings.  In theory, this should work
>>>> using the OCaml-conduit Vchan_lwt_unix mode, but in practise noone
>>>> has tried it yet.
>>>>
>>>> From the client side, it just needs a vchan call to read a certain
>>>> number of bytes and block if dom0 isn't supplying it with enough.
>>>>
>>>> -anil
>>>>
>>>> > On 19 Nov 2014, at 16:52, Hannes Mehnert <hannes@xxxxxxxxxxx> wrote:
>>>> >
>>>> > -----BEGIN PGP SIGNED MESSAGE-----
>>>> > Hash: SHA384
>>>> >
>>>> > Hello,
>>>> >
>>>> > in order to move OCaml-TLS onto Xen, there is one bit missing which I
>>>> > neither know in detail nor have the time to deal with.
>>>> > How to get entropy into a Mirage unikernel. The startup sequence is
>>>> > rather deterministic, and we don't want to require a RW object store
>>>> > to keep the seed (best practises in the UNIX world).
>>>> >
>>>> > Instead we would like to proxy /dev/urandom from dom0 into the
>>>> > unikernel to seed our random number generator.
>>>> >
>>>> > The interface is already there:
>>>> > https://github.com/mirage/mirage/blob/master/types/V1.mli#L75
>>>> > There is also an implementation for Xen, but this uses very weak
>>>> > entropy:
>>>> > https://github.com/mirage/mirage-entropy/tree/master/xen
>>>> >
>>>> >
>>>> > Some related work I found was virtio-rng
>>>> > (https://fedoraproject.org/wiki/Features/Virtio_RNG) which is supposed
>>>> > to work on Xen as well
>>>> > (http://wiki.xen.org/wiki/Virtio_On_Xen) -- but this might very likely
>>>> > be overengineered for our purposes.
>>>> >
>>>> > We (well, David) already have a state of the art random number
>>>> > generator implemented (Fortuna, design by Schneier + Ferguson) here:
>>>> > https://github.com/mirleft/ocaml-nocrypto/blob/master/src/fortuna.mli
>>>> >
>>>> >
>>>> > If someone could give that a try, it'd speed up to get mirage-tls into
>>>> > a usable state.
>>>> >
>>>> >
>>>> > Thanks,
>>>> >
>>>> > Hannes
>>>> > -----BEGIN PGP SIGNATURE-----
>>>> > Version: GnuPG v2
>>>> >
>>>> > iQIcBAEBCQAGBQJUbMqxAAoJELyJZYjffCjuaMMQAIH+9Q+t8Vr+x5DJFEX2nwFd
>>>> > xljxICmmrU2hNrKf0c3qv4b/tJvs9VrHLR93eM6meTu5aesyu8mJjMxJByowePBA
>>>> > mrL6y28SHnDrytfYDntWdNhlgCNch5d0HCsb3LmiE1TTY08ksn7BvDF1RiaTi1YJ
>>>> > 2BSsSrMnWdef+hd9U9yv4d8IqieBBK5JmAY9edkLzB9YYZySNkPVjnnWRb3UmNH1
>>>> > 0QvKnt+mO1T7R4GRix/YiSw97Too75u4hKP3ENSMrNZDax8OHu979F0JBxqfV8UI
>>>> > SsnNzvZ2Kve+1VfA/3ZJmvieBVPhq22EWkULJPXP3YMh1EQEG9UgIA3JxTg1WX6d
>>>> > rmLDjwOBDX35tnzC3MSRBFAbiNs+U0BE4DJINTRAdza74l4FRvpNaXrDY7DKvnXP
>>>> > CvETZ7xdnyO9a3G2zGrcU0kzC2XyePvwHa5ORJpuY8R3f3yvXDO761aQN27Npcs/
>>>> > GAvNCPp4iwyqYmEaGOQ5DzWYYP8dzo+ZRSXBWgIlnKNPOYof8Xn8gjQ06HyZ4ZrB
>>>> > Yg8kIeoSFxhsLL5Kr8tRkIn1I2ruy1/h9BKR0PmhIlPXBxhVkhfne2aUAjMqvKir
>>>> > DSu1AKssucGhbSe0XgLEV6pdGrf+5+mJnnpMZzoZvGXAZjyMDy9LTsUAkXP/CHYH
>>>> > SdxLRj15TylPbNNkJYLQ
>>>> > =SZPh
>>>> > -----END PGP SIGNATURE-----
>>>> >
>>>> > _______________________________________________
>>>> > MirageOS-devel mailing list
>>>> > MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
>>>> > http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel
>>>> >
>>>>
>>>>
>>>> _______________________________________________
>>>> MirageOS-devel mailing list
>>>> MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
>>>> http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel
>>>
>>> _______________________________________________
>>> MirageOS-devel mailing list
>>> MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
>>> http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel
>>>
>>>
>>

_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel