Re: [Xen-devel] LTTng Xen port : finally in a repository near you

[Mathieu Desnoyers <compudj@xxxxxxxxxxxxxxxxxx>]

Hello,

I made a working version of the LTTng tracer for xen-unstable for x86.
Here is the pointer to my repository so you can try it out :

hg clone http://ltt.polymtl.ca/cgi-bin/hgweb.cgi xen-unstable-lttng.hg

Basic usage :

(see lttctl-xen -h)

lttctl-xen -c

(in a different console)
lttd-xen -t /tmp/xentrace1

(in the 1st console)
lttctl-xen -s

(tracing is active)

lttctl-xen -q
lttctl-xen -r

lttd-xen should automatically quit after writing the last buffers as
soon as lttctl-xen -r is issued.

Then, you must copy the XML facilities :

(see the http://ltt.polymtl.ca > QUICKSTART to see how to install the
ltt-control package which contains the XML facilities in your system)

lttctl-xen -e -t /tmp/xentrace1

View in the visualiser : (see the QUICKSTART to see how to install it)

lttv -m textDump -t /tmp/xentrace1

(not tested yet) : to visualize a dom0 trace with the xen hypervisor
information, one would have to collect the dom0 kernel trace and the Xen
trace and open them together with :
lttv -m textDump -t /tmp/xentrace1 -t /tmp/dom0trace

The current Linux kernel instrumentation is for 2.6.20. A backport might
be needed to 2.6.18 if there is no proper Xen support in 2.6.20 (I have
not followed the recent developments).


Currently broken/missing :

- Ressources are not freed when the trace channels are destroyed. So you
  basically have to reboot between taking different traces.
- My code in the hypervisor complains to the console that subbuffers
  have not been fully read when the trace channels are destroyed. The
  error printing is just done too fast : lttd-xen is still there and
  reading the buffers at that point. It will get fixed with proper
  ressource usage tracking of both Xen and lttd-xen (same as the first
  point above).
- x86_64 not tested, powerpc local.h and ltt.h missing (should be ripped
  from my Linux kernel LTTng).


Cheers,

Mathieu



* Mathieu Desnoyers (compudj@xxxxxxxxxxxxxxxxxx) wrote:
> Hi,
> 
> My name is Mathieu Desnoyers, I am the current maintainer of the Linux Trace
> Toolkit project, known as LTTng. This is a tracer for the 2.6 Linux kernels
> oriented towards high performance and real-time applications.
> 
> I have read your tracing thread and I am surprised to see how much things
> you would like in a tracer are already implemented and tested in LTTng. I am
> currently porting my tracer to Xen, so I think it might be useful for you to
> know what it provides. My goal is to do not duplicate the effort and save
> everyone some time.
> 
> Here follows some key features of LTTng :
> 
> Architecture independant data types
> Extensible event records
> Self-describing traces
> Variable size records
> Fast (200 ns per event record)
> Highly reentrant
> Does not disable interrupts
> Does not take lock on the critical path
> Supports NMI tracing
> Analysis/visualization tool (LTTV)
> 
> Looking at the integration of the existing LTTng implementation into Xen, I
> came up with those two points for my Christmas whichlist :
> 
> Additionnal functionnalities that would be nice to have in Xen :
> 
> - RCU-style updates : would allow freeing the buffers without impact on 
> tracing.
>     * I guess I could currently use :
>       for_each_domain( d )
>         for_each_vcpu( d, v )
>           vcpu_sleep_sync(v);
>       I think it will have a huge impact on the system, but it would only be
>       performed before trace buffers free.
> 
> - Polling for data in Xen from a dom0 process.
>   Xentrace currently polls the hypervisor each 100ms to see if there is data
>   that needs to be consumed. Instead of an active polling, it would be nice to
>   use the dom0 OS capability to put a process to sleep while waiting for a
>   resource. It would imply creating a module, loaded in dom0, that would wait
>   for a specific virq coming from the Hypervisor to wake up such processes.
>   We could think of exporting a complete poll() interface through sysfs or
>   procfs that would be a directory filled with the resources exported from the
>   Hypervisor to dom0 (which could include wait for resource freed, useful when
>   shutting down a domU instead of busy looping). It would help dom0 to 
> schedule
>   other processes while a process is waiting for the Hypervisor.
> 
> 
> You might also be interested in looking at :
> - the website (http://ltt.polymtl.ca)
> - LTTng Xen port design document (this one is different from the one posted by
>   Jimi)
>   (http://ltt.polymtl.ca/svn/ltt/branches/poly/doc/developer/lttng-xen.txt)
> - OLS 2006 paper "The LTTng tracer : A Low Impact Performance and Behavior
>   Monitor for GNU/Linux"
>   (http://ltt.polymtl.ca/papers/desnoyers-ols2006.pdf)
> 
> 
> Questions and constructive comments are welcome.
> 
> Mathieu
> 
> 
> OpenPGP public key:              
> http://krystal.dyndns.org:8080/key/compudj.gpg
> Key fingerprint:     8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel
> 

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel