Re: Receiving Network Packets with Mirage/kFreeBSD

[Anil Madhavapeddy <anil@xxxxxxxxxx>]

On Thu, Aug 16, 2012 at 03:13:27PM +0200, PALI Gabor Janos wrote:
> 
> - After loading the built kernel module and starting up the evaluation of
>   the Netif.create function, Mirage asks for a list of available Ethernet
>   interfaces from the FreeBSD kernel and then assigns an lwt thread to each of
>   them by "plugging".
> 
> - Plugged interfaces are stored in a linked list and have their ng_ether(4)
>   hook (called from ether_input()) activated and pointed to
>   netif_ether_input().  At the same time, there is a shared ring buffer
>   created for each of them in Mirage then passed to the C function
>   responsible for administering the list of plugged interfaces,
>   caml_plug_vif().
> 
> - Shared ring buffers are created as Io_pages by allocating page-aligned,
>   contiguous, multi-page memory areas via FreeBSD's contigmalloc(9).  These
>   are directly accessible in Mirage as character arrays.
> 
> - Each shared ring buffer is currently of size 33 pages, and operates with
>   2048-byte slots.  The buffers start with a header that maintains all the
>   required meta information, like next position, available items, size of
>   stored items.

This all sounds spot on design-wise, by the way!  

> - Each packet arriving on any of the plugged interfaces is placed to the next
>   available slot of the corresponding shared ring buffer with m_copydata().
> 
> - In parallel with this in Mirage, the rx_poll function is run in loop that
>   polls for available packets in the shared ring buffer.
>   
> - When rx_poll finds unprocessed packets then it runs the user-specified
>   function on them, e.g. print the size of the packet in basic/netif.  It is
>   implemented by passing a view on the Io_page, i.e. without copying.  After
>   the user function has finished, the packet is removed from the ring.
> 
> - When no packets are available on the polled interface, rx_poll sleeps for a
>   millisecond.

What are your thoughts on a non-spinning interface here?  Can each ring
buffer also have an 'event channel' assigned to it, and then a single
select loop can block on all the event channels?  This isn't a huge deal
for the initial prototype where spinning is acceptable.

> Let me add that shared rings do not do locking at the moment, but if you are
> happy with the design I could proceed with implementing it.  (However, I am
> not completely sure how to share locks between OCaml and C.)

It's best to have a C interface which does an atomic test-and-set onto an
OCaml array, which can then be tested from within the Lwt/kFreeBSD
Activations code.  The OCaml code is guaranteed to not be running when
it's in a C-binding (assuming a single thread), so this should simplify
things.

-- 
Anil Madhavapeddy                                 http://anil.recoil.org