Re: [MirageOS-devel] An HTTP server with TLS

[Dave Scott <Dave.Scott@xxxxxxxxxx>]

> On 7 Jan 2015, at 17:12, Thomas Leonard <talex5@xxxxxxxxx> wrote:
> 
> On 7 January 2015 at 10:56, Anil Madhavapeddy <anil@xxxxxxxxxx> wrote:
>> On 7 Jan 2015, at 10:45, Thomas Leonard <talex5@xxxxxxxxx> wrote:
>>> 
>>> On 7 January 2015 at 10:42, Anil Madhavapeddy <anil@xxxxxxxxxx> wrote:
>>>> On 5 Jan 2015, at 09:53, Thomas Leonard <talex5@xxxxxxxxx> wrote:
>>>>> 
>>>>> I'd like to add TLS to my Mirage web server. What's the best way to do 
>>>>> this?
>>>>> 
>>>>> My Unikernel.Main functor currently takes a (H : Cohttp_lwt.Server)
>>>>> argument. I see that main.ml configures this using:
>>>>> 
>>>>> module Conduit1 = Conduit_mirage.Make(Stackv41)(Vchan1)
>>>>> module Http1 = HTTP.Make(Conduit1)
>>>>> 
>>>>> Can conduit deal with TLS for me? The conduit docs say "The reason
>>>>> this library exists is to provide a degree of abstraction from the
>>>>> precise SSL library used", which suggests that it should.
>>>> 
>>>> Conduit_mirage doesn't support this yet -- just Conduit_lwt_unix.
>>>> Before adding it in, I was waiting for xentropyd and the C bindings
>>>> to work, which should all be in the trees.  If we could now get a
>>>> mirage-skeleton example of a manual SSL server using the TCP/IP
>>>> stack directly, then the Conduit_mirage version won't be too far
>>>> behind.
>>> 
>>> tls/mirage/example has a direct example that works on Xen. I'm going
>>> to look at getting HTTPS support working now, unless you want to do it
>>> first.
>>> 
>> 
>> Go for it!  I'm taking a shot at pulling the OCaml runtime out of
>> mirage-platform at the moment.
> 
> OK. Could someone clarify the buffer-alignment rules for me again?
> 
> V1.mli says:
> 
> module type NETWORK = sig
>  type page_aligned_buffer
>  (** Abstract type for a page-aligned memory buffer *)
> 
> and
> 
> module type ETHIF = sig
>  type buffer
>  (** Abstract type for a memory buffer that may not be page aligned *)
> 
> tcpip's ethif.ml just passes the (non-aligned) buffer straight through
> to Netif, which seems wrong.
> 
> V1_LWT restricts the types with:
> 
> module type NETWORK = NETWORK
>   with type page_aligned_buffer = Io_page.t
> 
> module type ETHIF = ETHIF
>   with type buffer = Cstruct.t
> 
> io-page is a bit vague about what an Io_page.t is:
> 
> type t = (char, Bigarray.int8_unsigned_elt, Bigarray.c_layout) 
> Bigarray.Array1.t
> (** Type of memory blocks. *)
> 
> Io_page.get n returns "a memory block of [n] pages", so an Io_page.t
> isn't a single page of memory.
> 
> The actual problem I'm seeing with TLS on Xen is:
> 
> Invalid page: offset=2920, length=1245
> 
> This comes from Netif. The buffer underlying buffer is page aligned
> (it's allocated by Tls_mirage.conv_io), so I assume tcpip is splitting
> it at an unfortunate point.
> 
> It appears it was working before because HTTP_IO buffers its writes
> using tcpip's Channel module, which batches them into single IO pages.
> With TLS, these page-sized chunks don't go directly to TCP, but got
> via TLS instead.
> 
> So:
> 
> 1. What does "page-aligned memory buffer" really mean?

Itâs a bit of a mess atm :)

I think we need to write down our alignment requirements somewhere. I assume
they all come from the low-level drivers i.e. the higher-level layers donât
really care (is that true?)

Skimming though the netfront code I think that the protocol allows you to
grant a page and provide an offset within it, so you donât need to align
everything. You do need to split requests that cross page boundaries though.
One wrinkle is that if you donât trust the network backend (say itâs in
a driver domain with a dodgy wifi driver and has been compromised) then
you may not want to grant a page which happens to also contain some secret
data as well as your payload, since the untrustworthy backend can ignore the
offset and read the whole thing. Thinking about it, I suppose that would be
the driver-domain equivalent of heartblead: leaking random (Cstruct) buffers
on every packet.

IIRC the blkfront code expects the sectors to be page aligned.


> gnttab_stubs.c checks that the underlying Bigarray starts on a page boundary.
> 
> netif.ml checks that the cstruct's off + len <= page_size.

I think we discovered experimentally that netback didnât like it if it
crossed a boundary.

> 
> So from this, it seems that it means a page-aligned buffer no larger
> than a page.
> 
> 
> 2. Should Ethif split requests that cross page boundaries into
> multiple requests to Netif? Or do the APIs need changing?
> 
> 
> 3. Where should buffering happen? Between HTTP and TLS (as now), or
> between TLS and TCP?
> 
> 
> 4. Should we propagate buffer sizes backwards somehow, so that TCP can
> suggest to TLS to send data as TCP-sized chunks of data within a
> single Io_page?

Sometimes the drivers are able to process pages which youâve allocated yourself,
while in other cases theyâd prefer to do the allocation from a fixed pool.
For example blkfront with persistent grants, blkfront in userspace and vchan.
Perhaps we need to add some kind of allocator abstraction?

Cheers,
Dave

_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel