Xen project Mailing List

On Mon, Jan 12, 2015 at 7:13 PM, Anil Madhavapeddy <anil@xxxxxxxxxx> wrote:

On 12 Jan 2015, at 17:19, Thomas Leonard <talex5@xxxxxxxxx> wrote:
>
> On 8 January 2015 at 11:35, Thomas Leonard <talex5@xxxxxxxxx> wrote:
>> On 7 January 2015 at 17:35, Dave Scott <Dave.Scott@xxxxxxxxxx> wrote:
>>>
>>>> On 7 Jan 2015, at 17:12, Thomas Leonard <talex5@xxxxxxxxx> wrote:
>>>>
>>>> On 7 January 2015 at 10:56, Anil Madhavapeddy <anil@xxxxxxxxxx> wrote:
>>>>> On 7 Jan 2015, at 10:45, Thomas Leonard <talex5@xxxxxxxxx> wrote:
>>>>>>
>>>>>> On 7 January 2015 at 10:42, Anil Madhavapeddy <anil@xxxxxxxxxx> wrote:
>>>>>>> On 5 Jan 2015, at 09:53, Thomas Leonard <talex5@xxxxxxxxx> wrote:
>>>>>>>>
>>>>>>>> I'd like to add TLS to my Mirage web server. What's the best way to do this?
>>>>>>>>
>>>>>>>> My Unikernel.Main functor currently takes a (H : Cohttp_lwt.Server)
>>>>>>>> argument. I see that main.ml configures this using:
>>>>>>>>
>>>>>>>> module Conduit1 = Conduit_mirage.Make(Stackv41)(Vchan1)
>>>>>>>> module Http1 = HTTP.Make(Conduit1)
>>>>>>>>
>>>>>>>> Can conduit deal with TLS for me? The conduit docs say "The reason
>>>>>>>> this library exists is to provide a degree of abstraction from the
>>>>>>>> precise SSL library used", which suggests that it should.
>>>>>>>
>>>>>>> Conduit_mirage doesn't support this yet -- just Conduit_lwt_unix.
>>>>>>> Before adding it in, I was waiting for xentropyd and the C bindings
>>>>>>> to work, which should all be in the trees.Â If we could now get a
>>>>>>> mirage-skeleton example of a manual SSL server using the TCP/IP
>>>>>>> stack directly, then the Conduit_mirage version won't be too far
>>>>>>> behind.
>>>>>>
>>>>>> tls/mirage/example has a direct example that works on Xen. I'm going
>>>>>> to look at getting HTTPS support working now, unless you want to do it
>>>>>> first.
>>>>>>
>>>>>
>>>>> Go for it!Â I'm taking a shot at pulling the OCaml runtime out of
>>>>> mirage-platform at the moment.
>>>>
>>>> OK. Could someone clarify the buffer-alignment rules for me again?
>>>>
>>>> V1.mli says:
>>>>
>>>> module type NETWORK = sig
>>>> type page_aligned_buffer
>>>> (** Abstract type for a page-aligned memory buffer *)
>>>>
>>>> and
>>>>
>>>> module type ETHIF = sig
>>>> type buffer
>>>> (** Abstract type for a memory buffer that may not be page aligned *)
>>>>
>>>> tcpip's ethif.ml just passes the (non-aligned) buffer straight through
>>>> to Netif, which seems wrong.
>>>>
>>>> V1_LWT restricts the types with:
>>>>
>>>> module type NETWORK = NETWORK
>>>>Â with type page_aligned_buffer = Io_page.t
>>>>
>>>> module type ETHIF = ETHIF
>>>>Â with type buffer = Cstruct.t
>>>>
>>>> io-page is a bit vague about what an Io_page.t is:
>>>>
>>>> type t = (char, Bigarray.int8_unsigned_elt, Bigarray.c_layout) Bigarray.Array1.t
>>>> (** Type of memory blocks. *)
>>>>
>>>> Io_page.get n returns "a memory block of [n] pages", so an Io_page.t
>>>> isn't a single page of memory.
>>>>
>>>> The actual problem I'm seeing with TLS on Xen is:
>>>>
>>>> Invalid page: offset=2920, length=1245
>>>>
>>>> This comes from Netif. The buffer underlying buffer is page aligned
>>>> (it's allocated by Tls_mirage.conv_io), so I assume tcpip is splitting
>>>> it at an unfortunate point.
>>>>
>>>> It appears it was working before because HTTP_IO buffers its writes
>>>> using tcpip's Channel module, which batches them into single IO pages.
>>>> With TLS, these page-sized chunks don't go directly to TCP, but got
>>>> via TLS instead.
>>>>
>>>> So:
>>>>
>>>> 1. What does "page-aligned memory buffer" really mean?
>>>
>>> Itâs a bit of a mess atm :)
>>>
>>> I think we need to write down our alignment requirements somewhere. I assume
>>> they all come from the low-level drivers i.e. the higher-level layers donât
>>> really care (is that true?)
>>>
>>> Skimming though the netfront code I think that the protocol allows you to
>>> grant a page and provide an offset within it, so you donât need to align
>>> everything. You do need to split requests that cross page boundaries though.
>>> One wrinkle is that if you donât trust the network backend (say itâs in
>>> a driver domain with a dodgy wifi driver and has been compromised) then
>>> you may not want to grant a page which happens to also contain some secret
>>> data as well as your payload, since the untrustworthy backend can ignore the
>>> offset and read the whole thing. Thinking about it, I suppose that would be
>>> the driver-domain equivalent of heartblead: leaking random (Cstruct) buffers
>>> on every packet.
>>
>> It might be worth having Netif just copy everything to a pool of
>> pre-shared pages. That would save the time used granting and revoking
>> pages too, as well as improving security.
>>
>> Interestingly, it wouldn't add any performance overhead in this case
>> because copying the data in Netif would simply avoid the need for a
>> similar copy in TLS.
>
> I've now implemented this:
>
>Â https://github.com/mirage/mirage-net-xen/pull/17
>
> In the non-TLS case, my benchmark (sending UDP packets) increased in
> speed from 132 MB/s to 181 MB/s on Xen running under VirtualBox on my
> laptop.
>
> I was also able to download a 176M file over https from a Xen
> unikernel, at 16.2MB/s (before, it didn't work at all due to alignment
> problems).

Nice results!Â I was in two minds about these changes, since forcing a
copy at the lowest level is usually quite undesirable.Â However, I do
agree that a safer-by-default API is very important.Â I'd be happy to
have this copying/pregranting Netif interface be the default, and provide
an alternative Netif_zerocopy in the future that takes page-aligned
buffers only.Â The latter could introduce a type-safe zero copy API at
the same time.

The problem really has been our inability to change all the existing
Cstruct code to a new zero-copy API, and your solution neatly sidesteps
that.Â So if noone else has any objection to this diff, I'm happy to
merge it.

I think the argument to have a safe-by-default API is quite compelling. Let's merge this and work on our fancy zero-copy API later when we have the time.

-anil

_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel

Dave Scott

Re: [MirageOS-devel] An HTTP server with TLS