Re: [MirageOS-devel] Profiling Mirage on Xen

[Thomas Leonard <talex5@xxxxxxxxx>]

Interesting. I had to look at the things they did:

- Calculating the checksum at the same time as copying the data: we
now use checksum offloading to the hardware (or dom0), so we don't
calculate the checksum at all. For UDP checksum is optional anyway and
I think we skip it.

- Remembering which application sent/received the previous UDP packet
to avoid searching in the common case: might help, but we typically
only have one application in Mirage anyway. In my UDP tests, I only
testing sending. I need to look more closely at handling TCP acks.

- BSD treated UDP sockets as being connected to a particular endpoint,
and had to keep changing the address if the application specified the
destination with each packet. Mirage already has the correct API.

- Minimising privilege level changes and locking: these don't apply to
Mirage since it's single-threaded and runs at a fixed level.

- Pooling buffers to save on allocation. Could be useful.

In general though, the bottleneck seems to be the Xen API rather than
performance problems within Mirage. In all cases, we spend a lot of
time blocking and/or dealing with Xen grants.


On 2 September 2014 16:04, Jon Crowcroft <jon.crowcroft@xxxxxxxxxxxx> wrote:
> by the way, back in the day, someone did some work on UDP in BSD unix to get
> it to not go so slow (see
> A Faster UDP
> by partidge and pink
> http://dl.acm.org/citation.cfm?id=169943
> but i dont think anyone picked up on those optimisations in the Linux world
> (though obviously did for TCP) - not sure tho!
>
>
> On Thu, Aug 21, 2014 at 3:31 PM, Thomas Leonard <talex5@xxxxxxxxx> wrote:
>>
>> On 21 August 2014 15:09, Thomas Gazagnaire <thomas@xxxxxxxxxxxxxx> wrote:
>> >>> Cool -- it would be great to polish up a tool like this.
>> >>
>> >> Yes, I think there's a lot we could do here. I've written up the
>> >> profiling I've done so far here:
>> >>
>> >> http://roscidus.com/blog/blog/2014/08/15/optimising-the-unikernel/
>> >>
>> >> The graphs are quite interesting - if people familiar with the code
>> >> could explain what's going on (especially with the block device) that
>> >> would be great!
>> >
>> > I'm settting up a 4.01 switch with profiling on.
>>
>> Thanks!
>>
>> > Do you think the -fno-omit-frame-pointer configure options could also
>> > help to have more precise profiling information?
>>
>> I'm not sure it would help. The problems currently are:
>>
>> 1. We can't tell when a function returns until the next one is called.
>> 2. Tail calls look like the function returned first.
>> 3. For recursive calls, we can't tell which frame is the caller. (e.g.
>> foo calls foo and foo calls bar, but we don't know which foo
>> activation called bar).
>>
>> We could guess (3) by looking at the stack pointer. But it wouldn't
>> work for tail calls, since the stack doesn't grow and if we solved (1)
>> and (2) then we'd get (3) for free.
>>
>> I guess it wouldn't be hard to make the compiler call a trace function
>> on every return / before every tail call.
>>
>> The frame pointer would be useful if we didn't have tracing but wanted
>> a backtrace.
>>
>> > Thomas
>> >
>> >>
>> >> Overall, the changes increased the queuing service's download rate
>> >> from 2.46 MB/s to 7.24 MB/s, which is nice but still a bit
>> >> disappointing.
>> >>
>> >>
>> >> --
>> >> Dr Thomas Leonard        http://0install.net/
>> >> GPG: 9242 9807 C985 3C07 44A6  8B9A AE07 8280 59A5 3CC1
>> >> GPG: DA98 25AE CAD0 8975 7CDA  BD8E 0713 3F96 CA74 D8BA
>> >>
>> >> _______________________________________________
>> >> MirageOS-devel mailing list
>> >> MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
>> >> http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel
>> >
>>
>>
>>
>> --
>> Dr Thomas Leonard        http://0install.net/
>> GPG: 9242 9807 C985 3C07 44A6  8B9A AE07 8280 59A5 3CC1
>> GPG: DA98 25AE CAD0 8975 7CDA  BD8E 0713 3F96 CA74 D8BA
>>
>> _______________________________________________
>> MirageOS-devel mailing list
>> MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
>> http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel
>
>



-- 
Dr Thomas Leonard        http://0install.net/
GPG: 9242 9807 C985 3C07 44A6  8B9A AE07 8280 59A5 3CC1
GPG: DA98 25AE CAD0 8975 7CDA  BD8E 0713 3F96 CA74 D8BA

_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel