Mirage vs. FreeBSD Networking

[PÁLI Gábor János <pgj@xxxxxxx>]

Hi there,

I have been trying to make my Mirage/kFreeBSD modules work with a
vanilla (well, dtrace(1)- and kgdb(1)-enabled, VirtualBox-managed)
FreeBSD 9.1-RELEASE system, but apparently frames generated on the
localhost will not ever reach the fake NetGraph hooks I use to steal
them from the FreeBSD network stack [1][2].  This is probably due to
some optimizations to make all network traffic on the localhost not
touching the network card (which is em(4) in this case) at all.

The network layout of the machine currently is as follows (typical VBox NAT).

Destination  Gateway   Netif
default      10.0.2.2  em0
10.0.2.0/24  link#1    em0
10.0.2.15    link#1    lo0
127.0.0.1    link#2    lo0

And here is the network setup for the module to be tested, mir-ping
[3], for example.

ip-address: 10.0.2.215
ip-netmask: 255.255.255.0
ip-gateway: 10.0.2.2

That is, pinging 10.0.2.215 will not work in this setup: the Mirage
module will not see any incoming frame and the FreeBSD stack will
think that the host is down.

I have talked about this a little bit with Bjoern Zeeb and he
recommended to set a distinct MAC address for each of the Mirage
modules.  So (a) the frames will not looped back, (b) not all the
traffic will be hijacked by the Mirage stack (in other cases), only
the frames that are addressed to it, (c) this way multiple modules on
the system can safely co-exist, including the FreeBSD networking
stack.

However, this implies somehow a unique MAC address has to be assigned
to the module -- I did not find any way to do this in the mirari
configuration, hence I guess this has to be generated automatically
(and transparently).  Bjoern told me that the FreeBSD kernel also has
such routines implemented for other purposes, so I could piggyback on
this probably.

What do you think, are there any opinions, hints, recommendations on this?


[1] 
https://github.com/pgj/mirage-kfreebsd/blob/master/packages/mirage-platform/runtime/kernel/netif_stubs.c
[2] http://fxr.watson.org/fxr/source/net/if_ethersubr.c?v=FREEBSD91#L565
[3] https://github.com/mirage/mirage-skeleton/tree/master/ping