Re: [Xen-devel] [PATCH] docs/sphinx: todo/wishlist

[Juergen Gross <jgross@xxxxxxxx>]

On 23.07.19 15:38, Andrew Cooper wrote:
> On 23/07/2019 05:36, Juergen Gross wrote:
> > On 22.07.19 21:20, Andrew Cooper wrote:
> > > diff --git a/docs/misc/wishlist.rst b/docs/misc/wishlist.rst
> > > new file mode 100644
> > > index 0000000000..6cdb47d6e7
> > > --- /dev/null
> > > +++ b/docs/misc/wishlist.rst
> > > @@ -0,0 +1,53 @@
> > > +Development Wishlist
> > > +====================
> > > +
> > > +Remove xenstored's dependencies on unstable interfaces
> > > +------------------------------------------------------
> > > +
> > > +Various xenstored implementations use libxc for two purposes.  It
> > > would be a
> > > +substantial advantage to move xenstored onto entirely stable
> > > interfaces, which
> > > +disconnects it from the internal of the libxc.
> > > +
> > > +1. Foreign mapping of the store ring
> > > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > +
> > > +This is obsolete since :xen-cs:`6a2de353a9` (2012) which allocated
> > > grant
> > > +entries instead, to allow xenstored to function as a stub-domain
> > > without dom0
> > > +permissions.  :xen-cs:`38eeb3864d` dropped foreign mapping for
> > > cxenstored.
> > > +However, there are no OCaml bindings for libxengnttab.
> > > +
> > > +Work Items:
> > > +
> > > +* Minimal ``tools/ocaml/libs/xg/`` binding for ``tools/libs/gnttab/``.
> > > +* Replicate :xen-cs:`38eeb3864d` for oxenstored as well.
> > > +
> > > +
> > > +2. Figuring out which domain(s) have gone away
> > > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > +
> > > +Currently, the handling of domains is asymmetric.
> > > +
> > > +* When a domain is created, the toolstack explicitly sends an
> > > +  ``XS_INTRODUCE(domid, store mfn, store evtchn)`` message to
> > > xenstored, to
> > > +  cause xenstored to connect to the guest ring, and fire the
> > > +  ``@introduceDomain`` watch.
> > > +* When a domain is destroyed, Xen fires ``VIRQ_DOM_EXC`` which is
> > > bound by
> > > +  xenstored, rather than the toolstack.  xenstored updates its idea
> > > of the
> > > +  status of domains, and fires the ``@releaseDomain`` watch.
> > > +
> > > +Xenstored uses ``xc_domain_getinfo()``, to work out which domain(s)
> > > have gone
> > > +away, and only cares about the shutdown status.
> > > +
> > > +Furthermore, ``@releaseDomain`` (like ``VIRQ_DOM_EXC``) is a single-bit
> > > +message, which requires all listeners to evaluate whether the
> > > message applies
> > > +to them or not.  This results in a flurry of ``xc_domain_getinfo()``
> > > calls
> > > +from multiple entities in the system, which all serialise on the
> > > domctl lock
> > > +in Xen.
> > > +
> > > +Work Items:
> > > +
> > > +* Figure out how shutdown status can be expressed in a stable way
> > > from Xen.
> > > +* Figure out if ``VIRQ_DOM_EXC`` and ``@releaseDomain`` can be
> > > extended to
> > > +  carry at least a domid, to make domain shutdown scale better.
> > @releaseDomain (and @introduceDomain) can't be extended, we'd need to
> > add another watch path like @domainStatus/<domid>/<newState>. Xenstored
> > could advertise its capability to raise this watch in /tool/xenstored.
> I guess I was being a bit fast and loose with terminology.  I didn't
> intend to imply "literally modify @{introduce,release}Domain", as they
> are already fixed ABIs, but more to "compatibly build something which is
> better".
Okay.

> That scheme would work for improved @releaseDomain, but it wouldn't work
> for an improved introduce.  Introduce needs a single key to watch on,
> which hands back the domid so you don't need to go searching for it.
Yes, and? Its perfectly fine to set a watch firing if anything below
@domainStatus is changing.

> > As VIRQ_DOM_EXC is just an event I don't see how the domid could be
> > passed by it. I guess we'd need e.g. a shared memory area which the
> > domain registered for VIRQ_DOM_EXC could map and which would contain a
> > bitmap (one bit per domain). The hypervisor would set the bit on a
> > status change and fire VIRQ_DOM_EXC, xenstored would look for a set
> > bit, clear it and read the status of the related domain.
> The point here is to avoid using xc_domain_getinfo() in the first place,
> so there needs to be no "getting the status of the domain".
I'd guess a single xc_domain_getinfo() in the tools wouldn't be so
problematic. The caller would know the domid already, so no need to
query all domains.

> DOM_EXC is fired for domain_shutdown() only (but for reasons which
> escape me, fired twice).  Given that a domid is a 15 bit number, a
> bitmap of all domains does fit within a single 4k page.
Firing twice is needed: first time for disconnecting all backends
and the second time for cleaning up when the domain is completely
gone.

> All xenstored needs to know is that the domain shut down - nothing
> else.  Given a mapping such as the above, xenstored, on getting
> VIRQ_DOM_EXC would memcpy the page sideways, then loop over the page
> looking for bits which have become set, notify for each of those
> domains, and clear the bit in the page.
Right, that was the idea.

> > > +* Figure out if ``VIRQ_DOM_EXC`` would better be bound by the
> > > toolstack,
> > > +  rather than xenstored.
> > No, I don't think so. This would need another daemon.
> There is exactly one toolstack which is nominally daemonless, and it
> doesn't account for any substantial proportion of the Xen install-base,
> worldwide.
>
> The fact xl might not cope has no relevance to the technical discussion
> of whether the current asymmetric setup is sane or not, and whether it
> can, in principle, be improved.
Today we have either libvirtd or a per-domain xl instance running. So no
central piece of code for that purpose.


Juergen

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel