Re: xenstore - Suggestion of batching watch events

["Teddy Astie" <teddy.astie@xxxxxxxxxx>]

Hello,

Le 24/06/2025 à 16:53, Andriy Sultanov a écrit :
> Currently, as far as I am aware, the ability of xenstore clients to
> properly
> handle and detect batch updates is somewhat lacking. Transactions are not
> directly visible to the clients watching a particular directory - they will
> receive a lot of individual watch_event's once the transaction is
> committed,
> without any indication when such updates are going to end.
>
> Clients such as xenopsd from the xapi toolstack are reliant on xenstore to
> track their managed domains, and a flood of individual updates most often
> results in a flood of events raised from xenopsd to xapi (There are
> consolidation mechanisms implemented there, with updates getting merged
> together, but if xapi picks up update events from the queue quickly
> enough, it
> will only get more update events later)
>
> The need for batching is fairly evident from the fact that XenServer's
> Windows
> PV drivers, for example, adopted an ad-hoc "batch" optimization (not
> documented
> anywhere, of course), where some sequence of writes is followed by a
> write of
> the value "1" to "data/updated". This used to be honoured by xapi, which
> would
> not consider the guest agent update done until it received notice of such a
> "batch ended" update, but it caused xapi to miss updates that were not
> followed
> by such a write, so xapi now ignores this ad-hoc batching. One could
> imagine
> many workarounds here (for example, some sort of a mechanism where xenopsd
> stalls an update for a second to see if any more related updates show up
> and
> only then notifies xapi of it, with obvious trade-offs), but IMO it
> could be
> worth considering making this easier on the xenstore side for different
> use-cases.
>
> Suggestion:
> WATCH_EVENT's req_id and tx_id are currently 0. Could it be possible, for
> example, to modify this such that watch events coming as a result of a
> transaction commit (a "batch") have tx_id of the corresponding transaction
> and req_id of, say, 2 if it's the last such watch event of a batch and 1
> otherwise? Old clients would still ignore these values, but it would allow
> some others to detect if an update is part of a logical batch that
> doesn't end
> until its last event.
>

I find this a bit problematic as it adds assumption on a well-behaving
xenstored. What happens if you receive WATCH_EVENTs with only 1 (and no
2) ? Should the xenstore client assume that a WATCH_EVENT with 2 may
come at some point (lingering some arbitrary time), or still process it
in a singular way (like currently) ?

Without changing the protocol itself, we can improve the situation at
the xenstored/client level.

xenstore being architecturally a ring buffer (and a socket when
communicating directly with Dom0), we can batch multiples WATCH_EVENT in
a atomic way (up to ring size with ring buffer). In a way where the
client will be able to process multiple WATCH_EVENT without eventually
stalling.

(with ring buffer : write multiples xs_wire messages and only then
update ring index and raise event channel)

In userland, this could take the form as xenstored pushing multiples
messages through a single send(), so the client would be able to perform
all the recv() at once.

> Is this beyond the scope of what xenstored wants to do? From a first
> glance,
> this does not seem to introduce obvious unwanted information leaks
> either, but
> I could be wrong. I would love to hear if this is something that could be
> interesting to others and if this could be considered at all.
>
> Thank you!
>
>
>



Teddy Astie | Vates XCP-ng Developer

XCP-ng & Xen Orchestra - Vates solutions

web: https://vates.tech