Re: Preserving transactions accross Xenstored Live-Update

[Juergen Gross <jgross@xxxxxxxx>]

On 19.05.21 19:10, Julien Grall wrote:
> Hi Juergen,
>
> On 19/05/2021 13:50, Juergen Gross wrote:
> > On 19.05.21 14:33, Julien Grall wrote:
> > > On 19/05/2021 13:32, Julien Grall wrote:
> > > > Hi Juergen,
> > > >
> > > > On 19/05/2021 10:09, Juergen Gross wrote:
> > > > > On 18.05.21 20:11, Julien Grall wrote:
> > > > > > I have started to look at preserving transaction accross Live-update 
> > in
> > > > > > C Xenstored. So far, I managed to transfer transaction that 
> > > > > > read/write existing nodes.
> > > > > > Now, I am running into trouble to transfer new/deleted node within 
a 
> > > > > > transaction with the existing migration format.
> > > > > >
> > > > > > C Xenstored will keep track of nodes accessed during the transaction 
> > > > > > but not the children (AFAICT for performance reason).
> > > > > Not performance reasons, but because there isn't any need for that:
> > > > >
> > > > > The children are either unchanged (so the non-transaction node records
> > > > > apply), or they will be among the tracked nodes (transaction node
> > > > > records apply). So in both cases all children should be known.
> > > > In theory, opening a new transaction means you will not see any 
> > > > modification in the global database until the transaction has been 
> > > > committed. What you describe would break that because a client would 
> > > > be able to see new nodes added outside of the transaction.
> > > >
> > > > However, C Xenstored implements neither of the two. Currently, when 
> > > > a node is accessed within the transaction, we will also store the 
> > > > names of the current children.
> > > > To give an example with access to the global DB (prefixed with TID0) 
> > > > and within a transaction (TID1)
> > > >
> > > >      1) TID0: MKDIR "data/bar"
> > > >          2) Start transaction TID1
> > > >      3) TID1: DIRECTORY "data"
> > > >          -> This will cache thenode data
> > > >      4) TID0: MKDIR "data/foo"
> > > >          -> This will create 
"foo" in the global database
> > > >      5) TID1: MKDIR "data/fish"
> > > >          -> This will create 
"fish" inthe transaction
> > > >      5) TID1: DIRECTORY "data"
> > > >          -> This will only return "bar" and "fish"
> > > >
> > > > If we Live-Update between 4) and 5). Then we should make sure that 
> > > > "bar" cannot be seen in the listing by TID1.
> > > I meant "foo" here. Sorry for the confusion.
> > >
> > > > Therefore, I don't think we can restore the children using the 
> > > > global node here. Instead we need to find a way to transfer the list 
> > > > of known 
> > > > children within the transaction.
> > > >
> > > > As a fun fact, C Xenstored implements weirdly the transaction, so TID1 
> > > > will be able to access "bar" if it knows the name but not list it.
> > And this is the basic problem, I think.
> >
> > C Xenstored should be repaired by adding all (remaining) children of a
> > node into the TID's database when the list of children is modified
> > either globally or in a transaction. A child having been added globally
> > needs to be added as "deleted" into the TID's database.
> IIUC, for every modifications in the global database, we would need to 
> walk every single transactions and check whether a parent was accessed. 
> Am I correct?
Not really. When a node is being read during a transaction and it is
found in the global data base only, its gen-count can be tested for
being older or newer than the transaction start. If it is newer we can
traverse the path up to "/" and treat each parent the same way (so if
a parent is found in the transaction data base, the presence of the
child can be verified, and if it is global only, the gen-count can be
tested against the transaction again).

> If so, I don't think this is a workable solution because of the cost to 
> execute a single command.
My variant will affect the transaction internal reads only, and the
additional cost will be limited by the distance of the read node from
the root node.

> Is it something you plan to address differently with your rework of the 
DB?

Yes. I want to have the transaction specific variants of nodes linked to
the global ones, which solves this problem in an easy way.


Juergen

Attachment: OpenPGP_0xB0DE9DD628BF132F.asc
Description: OpenPGP public key

Attachment: OpenPGP_signature
Description: OpenPGP digital signature