Re: [MirageOS-devel] Mirage questions from writing a REST service

[Dave Scott <Dave.Scott@xxxxxxxxxx>]

On 8 Jul 2014, at 23:46, Anil Madhavapeddy <anil@xxxxxxxxxx> wrote:

> On 7 Jul 2014, at 10:08, Thomas Leonard <talex5@xxxxxxxxx> wrote:
> 
>> I'm writing a test Mirage service for queuing files. It's not using
>> ThomasG's merge-queues - I'm just trying the low-level FS stuff first.
>> 
>> Some problems I found:
>> 
>> 1. Running as a --unix process, I initially got a download speed of
>> 230 KB/s. Increasing the size of the chunks I was reading from the Fat
>> filesystem (a disk.img file) from 512 bytes to 1MB, I was able to
>> increase this to 3 MB/s. Both seem rather low. Apache on the same
>> machine gets 615 MB/s!
> 
> Yep, this is expected, since we explicitly don't have a buffer cache
> in the way and so everything you're doing is hitting the disk (and
> going through a not-very-optimized FAT implementation into the bargain).
> 
> The plan here is to define a buffer cache that's a functor over FS,
> and then apply it in config.ml (perhaps through a helper library function).

A functor over FS or over BLOCK should work.


>> I tried reproducing it with mirage-skeleton/static_website, but it
>> didn't want to know:
>> 
>> $ git clone https://github.com/mirage/mirage-skeleton.git
>> $ cd mirage-skeleton/static_website
>> $ FS=fat NET=socket mirage configure --unix
>> 
>> [ note: my actual test file is not all zeros, but for this email... ]
>> 
>> $ dd if=/dev/zero of=htdocs/data bs=1024 count=2
>> $ ./make-fat1-image.sh
>> $ make
>> $ ./mir-www
>> Manager: connect
>> Manager: configuring
>> Manager: socket config currently ignored (TODO)
>> Fatal error: exception Failure("fat2")
> 
> That's not a helpful error... any ideas ThomasG/Dave?

Unfortunately we don’t have pretty-printing of our mirage-types errors. This 
failure comes from the mirage autogen code:

```
(* Generated by Mirage (Wed, 9 Jul 2014 08:27:05 GMT). *)
…

module Fat2 = Fat.Fs.Make(Block)(Io_page)

let fat2 () =
  block1 () >>= function
  | `Error _ -> fail (Failure "fat2")
  | `Ok dev  -> Fat2.connect dev
```

Perhaps we should add “with sexp” to all our mirage-types errors, and then 
mirage could autogen calls to the right printer?

The failure might be that the disk is too small (2048 bytes?) I never 
implemented FAT12 so it’s good to go for larger disk sizes (16MiB+)

> 
>> 2. To be able to use the FS inferface, I had to specialise it like this:
>> 
>> module type FS = V1_LWT.FS with
>> type page_aligned_buffer = Cstruct.t and
>> type block_device_error = Fat.Fs.block_error
> 
> Ugh, the page_aligned_buffer remaining abstract in V1_LWT is definitely
> a bug:
> 
> (** FS *)
> module type FS = FS
>  with type 'a io = 'a Lwt.t
> 
> (all the other definitions override it).  Could you file a bug about
> this?  Best fixed in the 2.x branch.
> 
>> This means my unikernel only works with Fat. If I don't do this, I
>> can't report or log error messages. I guess this is also why the test
>> above gave such a useless error. Are there any plans to make error
>> handling easier?
> 
> Dave, what are your thoughts on this?  That is indeed a distinctly
> abstract-looking block device error :-)

I still find the error handling a bit awkward, particularly when you have 
layered devices and have wrapped errors. I think pretty-printing will 
definitely help though!

Cheers,
Dave

> 
>> 3. In my tags file, I use:
>> 
>> not <main.*>: warn(A)
>> 
>> Would be nice if the generated main.ml compiled without warnings (this
>> might be related to the previous point, as it's the error details that
>> it is ignoring).
> 
> Worth filing a bug for this too.
> 
>> 4. Can I get mirage to generate a Makefile that builds and runs my
>> unit-tests? I guess I can't run the unit-tests when in --xen mode,
>> because some modules have conflicting names, but it would be nice in
>> --unix mode at least.
> 
> Need some more details on the nature of these unit tests, as I don't
> understand what you mean by 'conflicting names'.  Can each unit test
> be compiled as a separate unikernel (so it'll work on Xen) via some
> config.ml code?
> 
> -anil
> 
> _______________________________________________
> MirageOS-devel mailing list
> MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
> http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel


_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel