[MirageOS-devel] Profiling Mirage on Xen

[Thomas Leonard <talex5@xxxxxxxxx>]

Has anyone made any tools for profiling on Xen?

I want to see why my network service on Xen/ARM only gets about 5 MB/s
(while a Linux guest gets about 45 MB/s).

I tried compiling with profiling on ("true: profile" in the _tags) and
wrote a __gnu_mcount_nc function to dump all the results to a buffer
[1], plus a script to turn the addresses back into symbols and guess
the nesting (a bit unreliable, as it doesn't tell you when the
function finishes).

Here's an example CSV of the output (the unikernel waits for a TCP
connection and then streams data to it as fast as it can):

  http://test.roscidus.com/static/sample-output.csv.bz2

I haven't checked it carefully to see if it's correct - this is just
an example of the kind of output. It shows the call graph and the
(cumulative) time spent in each function. Since it doesn't know the
end times, it assumes a function runs until one of its parents calls
something else.

Initially, the buffer filled up almost instantly with calls to
"stub_evtchn_test_and_clear". Looks like we call this once for each of
the 4096 channels every time we look for work. I'm not sure whether
this hurts performance much, but I reduced the number of channels to
10 to make the output readable.

The call graph is missing some nodes because I didn't compile e.g. Lwt
with profiling on. These functions have "?" shown for the time and no
parent. Is there an easy way to get OPAM to recompile everything for
profiling?

Does anyone have better tools for doing this? Especially something
that can link up async Lwt calls (or even track remote calls, like
Causeway [2]).

Anyway, I think it's quite cool to be able able to trace every kernel
call - even "irq_handler" shows up in the output!


[1] https://github.com/talex5/xen/commits/profiling
[2] http://wiki.erights.org/wiki/Causeway


-- 
Dr Thomas Leonard        http://0install.net/
GPG: 9242 9807 C985 3C07 44A6  8B9A AE07 8280 59A5 3CC1
GPG: DA98 25AE CAD0 8975 7CDA  BD8E 0713 3F96 CA74 D8BA

_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel