Re: [MirageOS-devel] Mirage questions from writing a REST service

[Anil Madhavapeddy <anil@xxxxxxxxxx>]

On 30 Jul 2014, at 09:18, Thomas Leonard <talex5@xxxxxxxxx> wrote:

> On 29 July 2014 14:04, David Scott <scott.dj@xxxxxxxxx> wrote:
>> Hi,
>> 
>> Very enjoyable post!
> 
> Thanks :-)
> 
>> On 29 Jul 2014 13:04, "Thomas Leonard" <talex5@xxxxxxxxx> wrote:
>>> 
>>> On 7 July 2014 10:08, Thomas Leonard <talex5@xxxxxxxxx> wrote:
>>>> I'm writing a test Mirage service for queuing files. It's not using
>>>> ThomasG's merge-queues - I'm just trying the low-level FS stuff first.
>>>> 
>>> 
>>> I did some more tests and wrote the results up here:
>>> 
>>>  http://roscidus.com/blog/blog/2014/07/28/my-first-unikernel/
>>> 
>>> In summary:
>>> 
>>> 1. Adding a block cache helped a bit, but it was still too slow.
>> 
>> I think a cache would be helpful for many applications. It's always a bit
>> painful to disable caching on linux-- O_DIRECT doesn't always work, and then
>> you have to align all your buffers. I like the uncached-by-default behaviour
>> of Mirage, but we should make it easier to turn caching back on.
> 
> Perhaps Mirage's block_of_file could take an argument to say whether
> you want it cached. For testing, you often want caching on because
> it's faster, while you want it off when benchmarking. For a real
> system you might want it either way.

I think the way to build these should be via a functor that takes
a BLOCK module type and returns a BLOCK module type, but with caching
inserted as part of its logic.  That can then be constructed directly
in the mirage combinator language using the @-> application.

> 
>>> 2. Dropping FAT and writing directly to the SD card helped a lot.
>> 
>> I think we should concentrate on a block backend for Irmin and label FAT as
>> not part of the Mirage "core". FAT will always be useful for things like
>> making a bootable disk image containing a kernel but it'll never be a
>> drop-in replacement for a modern Linux filesystem :-)
> 
> Are there any concrete plans for this yet? A block backend for Irmin
> would be ideal.

We're getting there -- Thomas (once he returns from his Indian functional
programming conference) will finish the Core_kernel removal from Irmin, and
then we can integrate the Baardskeerder B-Tree library from Arakoon.
That should give us everything we need to persist Irmin into a Xen block
store with no missing dependencies.

We've had Baardskeerder running on Mirage in the past (for the ASPLOS 2013
eval) but it has bitrotted against the newer APIs.   If anyone gets a chance
to resurrect it using the BLOCK interface, that would accelerate the Irmin
porting while Thomas is away.

> 
>>> 3. It's now fast enough for my use, but still slower than I was expecting.
>> 
>> It's very easy to make a small change which hammers performance. I think
>> I'll concentrate on writing  performance tests which can run from a CI when
>> I'm back.
> 
> That would be very useful. It might also show up places where ARM is
> extra slow (e.g. x86 has optimised TCP checksumming, so perhaps we
> should do something similar on ARM).

Dave, did your performance tests ever show up?  I finally have a machine 
setup on which I can run and publish the results regularly.

-anil
_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel