Re: [MirageOS-devel] Irmin API newbie questions

[Thomas Leonard <talex5@xxxxxxxxx>]

On 5 March 2015 at 11:56, Thomas Gazagnaire <thomas@xxxxxxxxxxxxxx> wrote:
>>>> - Irmin makes it easy to read data directly from a branch (tag), but
>>>> this is usually unsafe, since a tag can update while you're reading
>>>> it. Therefore, I think I always have to first get the head commit,
>>>> then create a new store for that revision, then read from that. It
>>>> might be safer if a branch just provided a way to get a commit, and
>>>> reads had to be done from a fixed commit.
>>>
>>> Hum, reading a tag should not be unsafe as the RW backend should ensure 
>>> that updates are atomic. See Get_unix.with_write_file[2] for example where 
>>> we use the fact that "rename" is the only operation guaranteed to be atomic 
>>> on POSIX filesystem.
>>
>> That's OK if you're only reading a single key, but usually you'll want
>> a consistent set of values, all from the same commit. e.g. if I do
>>
>>  I.read t "key1" >>= fun value1 ->
>>  I.read t "key2" >>= fun value2 ->
>>
>> then value1 and value2 will usually but not always be from the same
>> commit, which is an easy mistake to make. This would be safer:
>>
>>  Branch.head_commit t >>= fun c ->
>>  I.read c "key1" >>= fun value1 ->
>>  I.read c "key2" >>= fun value2 ->
>
> Ha I see, that you want is a transactional sequence of operations. In that 
> case you might want to use views [1]. All the read/write done on that view 
> are guaranteed to be isolated.

Currently, my main code reads a single revision like this:

I.of_head repo task_maker head >>= R.make

And R.make does:

  let make store =
    match I.branch (store "Get commit ID") with
    | `Tag _ -> failwith "Error: store is not fixed (use of_head)!"
    | `Head commit ->

Should I make a view here instead? It looks like View.of_path does a
load of copying. What I want is something that I can be sure will
never change. Neither views nor stores can ensure this I think (both
can be written to). Possibly I should do the "I.of_head" inside
R.make, but then I'd have to give it the repo (config) and task_maker
objects too, which is also ugly.

It seems strange, since the underlying Git model already provides
immutable data structures (blobs, trees, commits), but Irmin seems to
force me to treat everything as mutable.

> I still need to ensure that merging back in the store is really atomic (same 
> issue as with all the other high-level operations).

BTW, I read more of the storage spec and found:

"The use of the storage mutex to avoid race conditions is currently
considered by certain implementors to be too high a performance
burden, to the point where allowing data corruption is considered
preferable."

So some locking will be needed anyway. It would be good to implement
this only in the backend though, and have the rest of Irmin
automatically be safe.

> Thomas
>
> [1] http://mirage.github.io/irmin/Irmin.html#VALwith_hrw_view


-- 
Dr Thomas Leonard        http://0install.net/
GPG: 9242 9807 C985 3C07 44A6  8B9A AE07 8280 59A5 3CC1
GPG: DA98 25AE CAD0 8975 7CDA  BD8E 0713 3F96 CA74 D8BA

_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel