Re: [PATCH 10/11] tools/xenstore: drop use of tdb

[Juergen Gross <jgross@xxxxxxxx>]

On 19.06.23 20:22, Julien Grall wrote:
> Hi Juergen,
>
> I haven't looked at the code in details yet. But I have a few questions 
> regarding the commit message/
> On 30/05/2023 10:13, Juergen Gross wrote:
> > Today all Xenstore nodes are stored in a TDB data base. This data base
> > has several disadvantages:
> >
> > - it is using a fixed sized hash table, resulting in high memory
> >    overhead for small installations with only very few VMs, and a rather
> >    large performance hit for systems with lots of VMs due to many
> >    collisions
> Can you provide some concrete numbers and a setup in mind? This would help if 
> someone in the future says that they see the inverse and we need to rework the 
> logic.
The hash table size today is 7919 entries. This means that e.g. in case
of a simple desktop use case with 2 or 3 VMs probably far less than 10%
of the entries will be used (assuming roughly 100 nodes per VM). OTOH a
setup on a large server with 500 VMs would result in heavy conflicts in
the hash list with 5-10 nodes per hash table entry.

> > - Xenstore is only single-threaded, while TDB is designed to be fit
> >    for multi-threaded use cases, resulting in much higher code
> >    complexity than needed
> >
> > - special use cases of Xenstore are not possible to implement with TD
> By special use cases, are you referring to only the transaction below? If not 
> can you outline it?
I was thinking of splitting each node in the data base into a header, the
children list and the node contents. This would enable us to update only
the parts which are changed. Additionally read-only operations could just
use the instance of the node in the data base via a pointer instead of
allocating memory for a copy of the node.

For transactions I'd like to use a copy-on-write scheme with a per-node
list of the different transaction specific instances of a node (this needs
some more thought, though).

> >    in an effective way, while an implementation of a data base tailored
> >    for Xenstore could simplify some handling (e.g. transactions) a lot
> Do you have follow-up patches that would help to figuring out whether your new 
> interface makes sense?
Not yet.

> > So drop using TDB and store the nodes directly in memory making them
> > easily accessible. Use a hash-based lookup mechanism for fast lookup
> > of nodes by their full path.
> >
> > For now only replace TDB keeping the current access functions.
> Do you plan to have the rest of the work upstreamed for 4.18? Also, if for some 
> reasons, only this work will be merged. Will this have an impact on Xenstored 
> memory usage/performance?
Memory usage should go down, especially after deleting lots of entries
(AFAIK TDB will never free the unused memory again, it will just keep it
for the future).

Memory fragmentation might go up, though.

Performance might be better, too, as there is no need to realloc() the
memory when adding nodes.


Juergen

Attachment: OpenPGP_0xB0DE9DD628BF132F.asc
Description: OpenPGP public key

Attachment: OpenPGP_signature
Description: OpenPGP digital signature