Re: Python in Domain Configurations

[Ian Jackson <ijackson@xxxxxxxxxxxxxxxxxxxxxx>]

Elliott Mitchell writes ("Re: Python in Domain Configurations"):
> On Mon, Jul 31, 2023 at 05:59:55AM +0200, Marek Marczykowski-Górecki wrote:
> > So, IMHO reducing config file from a full python (like it used to be in
> > xend times) into a static file with well defined syntax was an
> > improvement. Lets not go backward.

I'm no longer working on this codebase, but since I've been CC'd:

I was one of the people who replaced the Python-based config parsing
with the current arrangements.  We didn't just do this because we were
replacing xend (whose use of Python as implementation language made it
appear convenient to just read and execute the configs as Python
code).

We did it for the reasons Marek gives.  It's true that the existing
format is not as well specified as it could be.  It was intended as a
plausible subset of Python literal syntax.  We chose that syntax to
preserve compatibility with the vast majority of existing config files
and to provide something familiar.  (And it seems we did achieve those
goals.)

The disk configuration syntax is particularly warty, but we inherited
much of that from the Python version.

If we had a free choice today, I might advocate for TOML.  But I don't
see any value in changing the concrete syntax now.

> > As for your original problem, IIUC you would like to add some data that
> > would _not_ be interpreted by libxl, right? For that you can use
> > comments with some specific marker for your script. This approach used
> > to work well for SysV init script, and in fact for a very similar use case
> > (ordering and dependencies, among other things).
> 
> That is /not/ the issue.  `xl` simply ignores any variables which it
> doesn't interpret (this is in fact a Bad Thing).

I forget, but isn't there some kind of scheme for warning about
unrecognised configuration options ?

>  I need to know what the limits to the syntax are.

I agree that it's not great that the syntax is not 100% documented.
The parser is in
  tools/libs/util/libxlu_cfg_y.y
  tools/libs/util/libxlu_cfg_l.l
I'm sure patches to improve the docs would be welcome.

Note that it is still a *subset* of Python, so if you wish to use a
Python interpreter to parse it in your own tooling, you're very
welcome to do so.

> Notice how many init scripts do `. /etc/default/<somefile>` to load
> configuration?  I'm thinking it would be very handy to use a similar
> technique to load domain.cfg files, with Python being the interpreter.

I don't think this is a good idea.  Both because I don't think the
functionality available in a Python interpreter should be available in
the libxl configuration, and because Python is a large and complex
dependency which we don't want to pull in here.

> I also think some portions of the domain.cfg format might work better
> with full Python syntax.  For example might it be handier to allow:
> 
> disk = [
>       {
>               'vdev': 'xvda',
>               'format': 'raw',
>               'access': 'rw',
>               'target': '/dev/disk/by-path/foo-bar-baz',
>       },
> ]

I agree that something like this would be nice.  I don't think it
should be done by importing Python.  These two files - the main part
of the existing parser - is only 183 loc including comments.
Extending it (and the support code in libxlu_cfg.c) to do dictionaries
as well as lists doesn't seem like it would make it too much bigger.

Thanks,
Ian.

-- 
Ian Jackson <ijackson@xxxxxxxxxxxxxxxxxxxxxx>   These opinions are my own.  

Pronouns: they/he.  If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.