Re: [MirageOS-devel] vchan hackers wanted for mirage-entropy

[Nicolas Ojeda Bar <no263@xxxxxxxxxxxxxxx>]

FYI

---------- Forwarded message ----------
From: Dave Scott <Dave.Scott@xxxxxxxxxx>
Date: Thu, Nov 20, 2014 at 2:55 PM
Subject: Re: ocaml-xenstore questions
To: Nicolas Ojeda Bar <no263@xxxxxxxxxxxxxxx>


Hi,

> On 20 Nov 2014, at 14:29, Nicolas Ojeda Bar <no263@xxxxxxxxxxxxxxx> wrote:
>
> Dear Dave,
>
> I have some questions and Anil suggested I emailed you about them.
>
> A little explanation:
>
> I am trying to use ocaml-xenstore to build a server in dom0 that will
> serve entropy from /dev/urandom to any unikernel that wants it via
> vchans.  I can already send the data using ocaml-vchan (manually
> setting the domid of the client) and now I want to use the xenstore to
> negotiate the client connections so that we do not need to enter the
> domid manually.  There is some code in conduit that seems to be what
> we need (Mirage_xenstore).

I think the conduit negotiation code works well enough as a prototype,
but itâs not secure enough for proper use because of the limitations
of the xenstore access control system.

In a Xen system the âtoolstackâ (xend, xl, xapi etc) uses its
privileged access to create backend and frontend directories, setting
the permissions and introducing the domains to each other. In the
conduit prototype we made it peer to peer (no toolstack) but this
meant we had to grant the equivalent to âchmod a+rwxâ to a xenstore
treeâ this means that anyone can interfere with anyone elseâs
connections.

https://github.com/mirage/ocaml-vchan/blob/master/lib_test/mirage/init-xenstore.sh#L6

The xenstore protocol (of which the access control stuff is part)
evolves very slowly because itâs a core part of the system. I think we
should explore changing that but it will take a while. In the meantime
I think you could solve the problem in a more (Xen-)traditional
fashion by creating a privileged daemon which runs in dom0 and acts
like the toolstack. It would

* watch for the arrival of new domains via the @introduceDomain watch
* create a vchan connection to each one directly

>
> Questions:
>
> - The latest ocaml-xenstore in opam is 1.2.5.  It seems to be quite
> out of date.  I pinned the current github repo but then mirage-xen
> stopped compiling.  Is this supposed to happen ? Do you know how to
> set things up to use the ocaml-xenstore trunk ?

The trunk contains interface changes which are still being developed.
When the interface changes are finished Iâll go around and update all
the clients (block driver, network driver, console driver etc). I
recommend avoiding this and sticking with 1.2.5 :-)


> - Mirage_xenstore in ocaml-conduit uses the OS.Xs module (a Xenstore
> client).  This module will not work in dom0 because the Unix backend
> of OS does not have a Xs submodule.  Anil thought that there was a
> unix implementation of a Xenstore client that could be used instead.
> Do you happen to know where to look for this ?

Have a look at the mirage console code:

https://github.com/mirage/mirage-console/blob/master/cli/main.ml

In particular this is a CLI which runs in dom0 and makes console
(rather than vchan) connections to other domains.

Watching xenstore for new domains is such a common requirement that
Iâd like to extend the xenstore API to include it. For now thereâs
some code here which can do it:

https://github.com/xapi-project/xenops/blob/master/src/xenstore_watch.ml#L48


> I believe that you will be by the Computer Lab later today, so I may
> pester you with more questions.
>
> Thank you very much!

Itâs also worth having a look at the qemu virtio-rng driver. Itâs got
some logic in there for rate-limiting â it would be good to understand
whether we need that too.

Good luck! :-)

Cheers,
Dave


>
> Best wishes,
> Nicolas

On Thu, Nov 20, 2014 at 11:36 AM, Anil Madhavapeddy <anil@xxxxxxxxxx> wrote:
> On 20 Nov 2014, at 09:57, Nick Hardiman <nick@xxxxxxxxxxxxxxxxxxxxxx> wrote:
>
>
> On 20 Nov 2014, at 05:02, david <unitedbiscuits@xxxxxxxxx> wrote:
>
>
>  i think it would be nice to assemble a blog post or set of instructions or
> something that describe the various ways of running xen in a development
> environment for mirage. (and deployment environments for that matter.)
>
>
> Speaking as an amateur idiot, I'd say there are missing steps for a Xen
> install that should be come before http://openmirage.org/wiki/install. I
> look forward to following the instructions.
>
>
> Agreed. It's become easier in recent years with the Ubuntu packaging
> improving, but it's by no means well documented.
>
> I just rebooted my VMWare Fusion Ubuntu setup with Xen and now the virtual
> graphics driver bombs out and hangs for some reason.  Ho hum...over to
> Virtualbox for me too I guess.
>
> Magnus: if you kick off the blog post about your setup, we can cut-and-paste
> that into a more centralised version to benefit other folk as well.  Nik's
> excellent work on the Vagrant setup should also be of use -- has anyone
> tried it? (it's at https://github.com/mirage/mirage-vagrant-vms)
>
> -anil
>
>
> _______________________________________________
> MirageOS-devel mailing list
> MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
> http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel
>

On Wed, Nov 19, 2014 at 5:24 PM, Nicolas Ojeda Bar
<no263@xxxxxxxxxxxxxxx> wrote:
> Yep, I got one so I should be set.
>
> Cheers,
> Nicolas
>
> On Wednesday, November 19, 2014, Anil Madhavapeddy <anil@xxxxxxxxxx> wrote:
>>
>> Great!  One minor annoyance with vchan is that it does require a Xen
>> machine to establish communications. If you don't have a Xen host, a
>> Cubieboard is the easiest way to get started in the short term, or a
>> Virtualbox setup.  I believe that Magnus is writing down the instructions
>> for his Virtualbox setup at the moment...
>>
>> -anil
>>
>> On 19 Nov 2014, at 17:19, Nicolas Ojeda Bar <no263@xxxxxxxxxxxxxxx> wrote:
>>
>> Hi Anil,
>>
>> I can try my hand at this; I need a break and wouldn't mind doing this
>> while I write the Ipv6 blog post.
>>
>> Cheers,
>> Nicolas
>>
>> On Wednesday, November 19, 2014, Anil Madhavapeddy <anil@xxxxxxxxxx>
>> wrote:
>>>
>>> Anyone got time for this?  Writing the dom0 proxy is pretty much a
>>> hello-world use of the vchan bindings.  In theory, this should work
>>> using the OCaml-conduit Vchan_lwt_unix mode, but in practise noone
>>> has tried it yet.
>>>
>>> From the client side, it just needs a vchan call to read a certain
>>> number of bytes and block if dom0 isn't supplying it with enough.
>>>
>>> -anil
>>>
>>> > On 19 Nov 2014, at 16:52, Hannes Mehnert <hannes@xxxxxxxxxxx> wrote:
>>> >
>>> > -----BEGIN PGP SIGNED MESSAGE-----
>>> > Hash: SHA384
>>> >
>>> > Hello,
>>> >
>>> > in order to move OCaml-TLS onto Xen, there is one bit missing which I
>>> > neither know in detail nor have the time to deal with.
>>> > How to get entropy into a Mirage unikernel. The startup sequence is
>>> > rather deterministic, and we don't want to require a RW object store
>>> > to keep the seed (best practises in the UNIX world).
>>> >
>>> > Instead we would like to proxy /dev/urandom from dom0 into the
>>> > unikernel to seed our random number generator.
>>> >
>>> > The interface is already there:
>>> > https://github.com/mirage/mirage/blob/master/types/V1.mli#L75
>>> > There is also an implementation for Xen, but this uses very weak
>>> > entropy:
>>> > https://github.com/mirage/mirage-entropy/tree/master/xen
>>> >
>>> >
>>> > Some related work I found was virtio-rng
>>> > (https://fedoraproject.org/wiki/Features/Virtio_RNG) which is supposed
>>> > to work on Xen as well
>>> > (http://wiki.xen.org/wiki/Virtio_On_Xen) -- but this might very likely
>>> > be overengineered for our purposes.
>>> >
>>> > We (well, David) already have a state of the art random number
>>> > generator implemented (Fortuna, design by Schneier + Ferguson) here:
>>> > https://github.com/mirleft/ocaml-nocrypto/blob/master/src/fortuna.mli
>>> >
>>> >
>>> > If someone could give that a try, it'd speed up to get mirage-tls into
>>> > a usable state.
>>> >
>>> >
>>> > Thanks,
>>> >
>>> > Hannes
>>> > -----BEGIN PGP SIGNATURE-----
>>> > Version: GnuPG v2
>>> >
>>> > iQIcBAEBCQAGBQJUbMqxAAoJELyJZYjffCjuaMMQAIH+9Q+t8Vr+x5DJFEX2nwFd
>>> > xljxICmmrU2hNrKf0c3qv4b/tJvs9VrHLR93eM6meTu5aesyu8mJjMxJByowePBA
>>> > mrL6y28SHnDrytfYDntWdNhlgCNch5d0HCsb3LmiE1TTY08ksn7BvDF1RiaTi1YJ
>>> > 2BSsSrMnWdef+hd9U9yv4d8IqieBBK5JmAY9edkLzB9YYZySNkPVjnnWRb3UmNH1
>>> > 0QvKnt+mO1T7R4GRix/YiSw97Too75u4hKP3ENSMrNZDax8OHu979F0JBxqfV8UI
>>> > SsnNzvZ2Kve+1VfA/3ZJmvieBVPhq22EWkULJPXP3YMh1EQEG9UgIA3JxTg1WX6d
>>> > rmLDjwOBDX35tnzC3MSRBFAbiNs+U0BE4DJINTRAdza74l4FRvpNaXrDY7DKvnXP
>>> > CvETZ7xdnyO9a3G2zGrcU0kzC2XyePvwHa5ORJpuY8R3f3yvXDO761aQN27Npcs/
>>> > GAvNCPp4iwyqYmEaGOQ5DzWYYP8dzo+ZRSXBWgIlnKNPOYof8Xn8gjQ06HyZ4ZrB
>>> > Yg8kIeoSFxhsLL5Kr8tRkIn1I2ruy1/h9BKR0PmhIlPXBxhVkhfne2aUAjMqvKir
>>> > DSu1AKssucGhbSe0XgLEV6pdGrf+5+mJnnpMZzoZvGXAZjyMDy9LTsUAkXP/CHYH
>>> > SdxLRj15TylPbNNkJYLQ
>>> > =SZPh
>>> > -----END PGP SIGNATURE-----
>>> >
>>> > _______________________________________________
>>> > MirageOS-devel mailing list
>>> > MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
>>> > http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel
>>> >
>>>
>>>
>>> _______________________________________________
>>> MirageOS-devel mailing list
>>> MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
>>> http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel
>>
>> _______________________________________________
>> MirageOS-devel mailing list
>> MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
>> http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel
>>
>>
>

_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel