Re: [Xen-devel] [PATCH] docs/sphinx: todo/wishlist

[Andrew Cooper <andrew.cooper3@xxxxxxxxxx>]

On 23/07/2019 05:36, Juergen Gross wrote:
> On 22.07.19 21:20, Andrew Cooper wrote:
>> diff --git a/docs/misc/wishlist.rst b/docs/misc/wishlist.rst
>> new file mode 100644
>> index 0000000000..6cdb47d6e7
>> --- /dev/null
>> +++ b/docs/misc/wishlist.rst
>> @@ -0,0 +1,53 @@
>> +Development Wishlist
>> +====================
>> +
>> +Remove xenstored's dependencies on unstable interfaces
>> +------------------------------------------------------
>> +
>> +Various xenstored implementations use libxc for two purposes.  It
>> would be a
>> +substantial advantage to move xenstored onto entirely stable
>> interfaces, which
>> +disconnects it from the internal of the libxc.
>> +
>> +1. Foreign mapping of the store ring
>> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> +
>> +This is obsolete since :xen-cs:`6a2de353a9` (2012) which allocated
>> grant
>> +entries instead, to allow xenstored to function as a stub-domain
>> without dom0
>> +permissions.  :xen-cs:`38eeb3864d` dropped foreign mapping for
>> cxenstored.
>> +However, there are no OCaml bindings for libxengnttab.
>> +
>> +Work Items:
>> +
>> +* Minimal ``tools/ocaml/libs/xg/`` binding for ``tools/libs/gnttab/``.
>> +* Replicate :xen-cs:`38eeb3864d` for oxenstored as well.
>> +
>> +
>> +2. Figuring out which domain(s) have gone away
>> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> +
>> +Currently, the handling of domains is asymmetric.
>> +
>> +* When a domain is created, the toolstack explicitly sends an
>> +  ``XS_INTRODUCE(domid, store mfn, store evtchn)`` message to
>> xenstored, to
>> +  cause xenstored to connect to the guest ring, and fire the
>> +  ``@introduceDomain`` watch.
>> +* When a domain is destroyed, Xen fires ``VIRQ_DOM_EXC`` which is
>> bound by
>> +  xenstored, rather than the toolstack.  xenstored updates its idea
>> of the
>> +  status of domains, and fires the ``@releaseDomain`` watch.
>> +
>> +Xenstored uses ``xc_domain_getinfo()``, to work out which domain(s)
>> have gone
>> +away, and only cares about the shutdown status.
>> +
>> +Furthermore, ``@releaseDomain`` (like ``VIRQ_DOM_EXC``) is a single-bit
>> +message, which requires all listeners to evaluate whether the
>> message applies
>> +to them or not.  This results in a flurry of ``xc_domain_getinfo()``
>> calls
>> +from multiple entities in the system, which all serialise on the
>> domctl lock
>> +in Xen.
>> +
>> +Work Items:
>> +
>> +* Figure out how shutdown status can be expressed in a stable way
>> from Xen.
>> +* Figure out if ``VIRQ_DOM_EXC`` and ``@releaseDomain`` can be
>> extended to
>> +  carry at least a domid, to make domain shutdown scale better.
>
> @releaseDomain (and @introduceDomain) can't be extended, we'd need to
> add another watch path like @domainStatus/<domid>/<newState>. Xenstored
> could advertise its capability to raise this watch in /tool/xenstored.

I guess I was being a bit fast and loose with terminology.  I didn't
intend to imply "literally modify @{introduce,release}Domain", as they
are already fixed ABIs, but more to "compatibly build something which is
better".

That scheme would work for improved @releaseDomain, but it wouldn't work
for an improved introduce.  Introduce needs a single key to watch on,
which hands back the domid so you don't need to go searching for it.

>
> As VIRQ_DOM_EXC is just an event I don't see how the domid could be
> passed by it. I guess we'd need e.g. a shared memory area which the
> domain registered for VIRQ_DOM_EXC could map and which would contain a
> bitmap (one bit per domain). The hypervisor would set the bit on a
> status change and fire VIRQ_DOM_EXC, xenstored would look for a set
> bit, clear it and read the status of the related domain.

The point here is to avoid using xc_domain_getinfo() in the first place,
so there needs to be no "getting the status of the domain".

DOM_EXC is fired for domain_shutdown() only (but for reasons which
escape me, fired twice).  Given that a domid is a 15 bit number, a
bitmap of all domains does fit within a single 4k page.

All xenstored needs to know is that the domain shut down - nothing
else.  Given a mapping such as the above, xenstored, on getting
VIRQ_DOM_EXC would memcpy the page sideways, then loop over the page
looking for bits which have become set, notify for each of those
domains, and clear the bit in the page.

>
>> +* Figure out if ``VIRQ_DOM_EXC`` would better be bound by the
>> toolstack,
>> +  rather than xenstored.
>
> No, I don't think so. This would need another daemon.

There is exactly one toolstack which is nominally daemonless, and it
doesn't account for any substantial proportion of the Xen install-base,
worldwide.

The fact xl might not cope has no relevance to the technical discussion
of whether the current asymmetric setup is sane or not, and whether it
can, in principle, be improved.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel