Re: I am confused by Mirage_kv.S.batch

[Thomas Gazagnaire <thomas@xxxxxxxxxxxxxx>]

Hi Mindy :-)

The semantics intend to match the “commit” operation of Irmin, e.g. a commit 
will ever read what is in the current checkout (and/or what have been written 
in that commit since the beginning of the transaction). The important bits is 
that it won’t be able to read writes done by concurrent writers.

It’s obviously not super easy to implement if you do not have access to a 
transactional file-system. In that case, I won’t worry too much about it; the 
only important thing is durability, e.g. the data should be written (and 
flushed) to disk once the operation has completed. We could probably edit that 
docstring to remove the sentence about “reading inside a batch”: I don’t think 
any user of Mirage_kv is relying on this behaviour at the moment.

Best,
Thomas

> On 2 Jan 2022, at 23:13, Mindy <mindy@xxxxxxxxxxxxxxxxxxx> wrote:
> 
> Hi all,
> 
> I've been working on a block-backed Mirage_kv.S implementation inspired by 
> (and largely interoperable with) littlefs[1] . I'm approaching readiness for 
> release, but there remains one large unimplemented piece of Mirage_kv.S : 
> `batch`. The relevant excerpt of mirage_kv.mli is reproduced below:
> 
> ```
> 
>   val batch: t -> ?retries:int -> (t -> 'a Lwt.t) -> 'a Lwt.t
>   (** [batch t f] run [f] in batch. Ensure the durability of
>      operations.
> 
>       Since a batch is applied at once, the readings inside a batch
>      will return the state before the entire batch. Concurrent
>      operations will not affect other ones executed during the batch.
> 
>       Batch applications can fail to apply if other operations are
>      happening concurrently. In case of failure, [f] will run again
>      with the most recent version of [t]. The result is
>      [Error `Too_many_retries] if [f] is run for more then [retries] attemps
>      (default is [13]). *)
> ```
> 
> I confess to being uncertain how to implement the "readings inside a batch" 
> portion of this. Given an arbitrary function `t -> 'a Lwt.t`, how is the 
> implementer expected to differentiate between operations within such a 
> function, possibly implemented as a series of sequential binds?
> 
> Currently my implementation simply runs the given function against the 
> filesystem, but this is clearly semantically wrong given the explanation in 
> the docstring.
> 
> I see `wodan` implements this by differentiating between a `t` used for read 
> operations and one used for write operations[2] , but I lack the 
> understanding of `wodan` necessary to understand how this can work for a `t 
> -> 'a Lwt.t` that mixes read and write operations, which I think the 
> docstring implies to be an expected use of `batch`.
> 
> Thanks for your help,
> 
> Mindy
> 
> [1] https://github.com/littlefs-project/littlefs
> 
> [2] 
> https://github.com/mirage/wodan/blob/main/src/wodan-irmin/wodan_irmin.ml#L345
> 
>