[MirageOS-devel] Fwd: command line arguments for unikernels

[Nicolas Ojeda Bar <no263@xxxxxxxxxxxxxxx>]

Hi list,

I am forwarding some interesting discussion we have been having with
Magnus & ThomasG about how to improve the passing of arguments to
unikernels, both at compile/configure time and at *run*-time.  The
objective is to get rid of all the boilerplate code floating around
for configuring things like the network stack (static vs dhcp, ip
addresses, direct vs socket, ...), filesystems, etc.  The current
state of affairs often requires to manually modify config.ml or,
worse, the generated main.ml file and recompile.

There is a rough implementation of a possible approach available (see
the very first email for instructions & an example).

Any comments, thoughts, opinions, etc. would be greatly appreciated.

Cheers
Nicolas

---------- Forwarded message ----------
From: Thomas Gazagnaire <thomas@xxxxxxxxxxxxxx>
Date: Fri, Nov 28, 2014 at 4:03 PM
Subject: Re: command line arguments for unikernels
To: Magnus Skjegstad <magnus@xxxxx>
Cc: Nicolas Ojeda Bar <no263@xxxxxxxxxxxxxxx>


btw, would be better to have this interesting discussion directly on
the mirage list.

> On 28 Nov 2014, at 12:01, Magnus Skjegstad <magnus@xxxxx> wrote:
>
> On Thu, 27 Nov 2014, at 22:59, Thomas Gazagnaire wrote:
>> about the env: kind of agree, but you have some cases where it is useful
>> (OPAMYES and OPAMDEBUG are good examples for opam). But I agree that it
>> can become confusing, that's why I proposed to clearly display to the
>> user at configure/runtime where the configuration value comes from. Or
>> this can be left optional.
>
> Hm, but even if it is clearly shown where the parameters are read from
> in the console, it would require someone to check the run time output to
> make sure the configuration is correct. The configuration could also
> unexpectedly change if the user who starts it makes changes to his/her
> environment for other reasons.
>
> Instead of adding optional env parameters in Mirage tool, we could just
> use the shell when this functionality is needed. For example, you could
> set --val=$SOMETHING on the command line to clearly show which
> environment variable is in use. You could use default values in bash if
> the environment variable is not set: --val=${IP-10.0.0.1} will use $IP
> if set, otherwise 10.0.0.1.
>
> Using the shell would also allow us to use environment variables with
> different names than the parameters, which could avoid conflicts - and
> avoid complexity in the Mirage tool :)
>
>> Would be indeed quite neat to have "mirage configure --help" to work,
>> this means adding a proper cmdliner argument for every dynamic parameter,
>> and maybe add a new function in CONFIGURABLE to gather all such
>> parameters.
>>
>> And yes, an clear error at runtime if a mandatory dynamic parameter is
>> missing is a good idea :-) I think cmdliner combinators[1] can be a great
>> source of inspiration for that.
>>
>> Thomas
>> [1] http://erratique.ch/software/cmdliner/doc/Cmdliner.Arg.html
>>
>>> On 27 Nov 2014, at 19:50, Nicolas Ojeda Bar <no263@xxxxxxxxxxxxxxx> wrote:
>>>
>>> Hi Magnus,
>>>
>>> I already pushed to nojb/mirage so you can take a look if you want.
>>>
>>> I agree on environment variables - I will take that out.  I am looking
>>> to modify the format of the
>>> command line arguments to --key=value and also to enable `mirage
>>> configure` to "know" which
>>> parameters need to be set/have been set (as Thomas suggested). And
>>> yes, any parameter that
>>> has been declared `dynamic` can be given or overridden at run-time.
>>>
>>> Thanks very much for the feedback!
>>>
>>> Cheers,
>>> Nicolas
>>>
>>> On Thu, Nov 27, 2014 at 7:37 PM, Magnus Skjegstad <magnus@xxxxx> wrote:
>>>> Great! This looks very good!
>>>>
>>>> I agree with Thomas that it would be great to have --xxx for every
>>>> parameter during compile time as well. Makes it easier to remember :-)
>>>> Can you give parameters during run time that wasn't specified during
>>>> compile time?
>>>>
>>>> Not related to the code, but may be useful to mention in the docs:  With
>>>> xl in Xen you can override settings in the .xl file from the command
>>>> line, so if you want to use dynamic settings you don't have to rewrite
>>>> the config file every time:
>>>>
>>>> xl create unikernel.xl 'extra="ip=192.168.0.1 .... "'
>>>>
>>>> I am not a fan of using environment variables to configure a program
>>>> though, so personally, I would prefer it those were left out of it ;-)
>>>> (and you could always do --ip=$IP). The problem is that old env
>>>> variables can change how the program works unintentionally - which may
>>>> be ok on a dev server, but I'm not sure I would trust the environment to
>>>> always be correct if I wanted to run unikernels automatically from a
>>>> script somewhere for example. With custom params the probability of name
>>>> collisions with existing env variables would also be high.
>>>>
>>>> Regarding breakage, wouldn't it be better if the unikernel exited with
>>>> an informative error in console/log when the run time parameters were
>>>> wrong/unparsable? For my use case, a default value that enabled the
>>>> unikernel to run, but didn't actually work would be worse than if it
>>>> exited early...
>>>>
>>>> Looking forward to seeing the code! :-)
>>>>
>>>> Magnus
>>>>
>>>> On Thu, 27 Nov 2014, at 18:49, Thomas Gazagnaire wrote:
>>>>> Awesome!
>>>>>
>>>>> but You forgot to push your code to nojb/mirage :p
>>>>>
>>>>> I have few comments:
>>>>> - would be even better to have --ip=xxx --netmaks=.... for both the
>>>>> mirage command-line and the unix binary command-line.
>>>>> - runtime params are cool, but I guess they always need a default
>>>>> otherwise we can see a breakage very late (sometime too late)
>>>>> - would be nice to display to the user which config variables are set at
>>>>> configuration time (and where they are picked-up, the env or the
>>>>> command-line), and which one remains to be set at runtime.
>>>>>
>>>>> Apart that, this seems pretty cool :-)
>>>>>
>>>>> Thanks!
>>>>> Thomas
>>>>>
>>>>>> On 27 Nov 2014, at 18:38, Nicolas Ojeda Bar <no263@xxxxxxxxxxxxxxx> 
>>>>>> wrote:
>>>>>>
>>>>>> Hi guys,
>>>>>>
>>>>>> I have made some progress on the idea we discussed yesterday on how to
>>>>>> specify command line arguments to pass to the unikernels.  I thought I
>>>>>> could get some feedback from you before sending it to the list.
>>>>>>
>>>>>> There is documentation near the top of `mirage.mli` (see below for
>>>>>> which branch) that you can read to try to make sense of the design.
>>>>>>
>>>>>> How to use it:
>>>>>>
>>>>>> There is a new type, 'a Mirage.Param.t, which represents a parameter
>>>>>> (given at compile-time, run-time or hard-coded) of type 'a. There is
>>>>>> no way to retrieve the contents of a value of this type because the
>>>>>> contents may not be available until run-time.  What this type does is
>>>>>> to generate code in `main.ml` representing values given at
>>>>>> compile-time or generate code to extract the values from the kernel
>>>>>> parameters passed at run-time (in the 'extra' field of .xl file for
>>>>>> xen, in the command line for unix).
>>>>>>
>>>>>> The combinators in Mirage need to be modified slightly to take 'a
>>>>>> Param.t's instead of values of type 'a.  This is a small change in
>>>>>> general.
>>>>>>
>>>>>> Example: https://github.com/nojb/mirage-better-params-ex
>>>>>>
>>>>>> In this example we can specify which tap device, ip address, netmask,
>>>>>> and gateway addresses to use in any of the following ways:
>>>>>>
>>>>>> At compile-time:
>>>>>>
>>>>>> - mirage configure --unix --extra 'tap=tap0 ip=0.0.0.0 netmask=0.0.0.0
>>>>>> gateways=0.0.0.0,0.0.0,1'
>>>>>> - tap=tap0 mirage configure --unix --extra 'ip=0.0.0.0 netmask=0.0.0.0
>>>>>> gateways=0.0.0.0,0.0.0,1'
>>>>>> - tap=tap0 ip=0.0.0.0 mirage configure --unix --extra 'netmask=0.0.0.0
>>>>>> gateways=0.0.0.0,0.0.0,1'
>>>>>>
>>>>>> etc.
>>>>>>
>>>>>> At run-time:
>>>>>>
>>>>>> - (unix) ./mir-test tap=tap0 ip=0.0.0.0 netmask=0.0.0.0 
>>>>>> gateways=0.0.0.0,0.0.0,1
>>>>>> - (xen) by adding the line
>>>>>> extra = 'tap=tap0 ip=0.0.0.0 netmask=0.0.0.0 gateways=0.0.0.0,0.0.0,1'
>>>>>> to the generated `.xl` file.
>>>>>>
>>>>>> To try it out:
>>>>>> ```
>>>>>> opam pin add mirage-xen 
>>>>>> https://github.com/nojb/mirage-platform#better-params
>>>>>> opam pin add mirage-unix 
>>>>>> https://github.com/nojb/mirage-platform#better-params
>>>>>> opam pin add mirage https://github.com/nojb/mirage#better-params
>>>>>> ```
>>>>>>
>>>>>> Thoughts ? Comments ?
>>>>>>
>>>>>> Cheers,
>>>>>> Nicolas
>>>>>
>>

_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel