Re: [Xen-devel] LTTng-Xen Buffer shared between the hypervisor and a dom0 process

[Mathieu Desnoyers <compudj@xxxxxxxxxxxxxxxxxx>]

* Keir Fraser (keir@xxxxxxxxxxxxx) wrote:
> On 8/3/07 09:26, "Mathieu Desnoyers" <compudj@xxxxxxxxxxxxxxxxxx> wrote:
> 
> > Not exactly : when xen wants to end writing to its buffers, it disables
> > writing, does a subbuffer switch and sets the buffer "finalize" flag to
> > 1.  It then sends a VIRQ to lttd-xen. lttd-xen reads the last subbuffers
> > (using a get_cpu/put_cpu and get_facilities/put_facilities cmd of the
> > ltt hypercall to select the offset to read and then reads the buffers)
> > and is then ready to release the buffers. At that specific point, I
> > would like all the trace information (xmalloc'd and xenheap shared) to
> > be freed. But I would also like it to be freed if lttd is killed (so
> > when file descriptors and memory maps are freed).
> 
> This latter part sounds kind of tricky to do cleanly. How do we distinguish
> between refcount being zero because lttd-xen has died, versus refcount being
> zero because lttd-xen hasn't yet mapped the buffers?

We don't have to. The cycle to take a trace goes like this :

lttctl-xen -c (allocate trace buffers)
lttd-xen -t /tmp/mytrace (maps the buffers, waits for data)
lttctl-xen -s (flip the "tracing active" bit, start recording)
...
lttctl-xen -q (turn off tracing)
(we can do multiple start/stop if needed)
lttctl-xen -r (finalize buffers, signal lttd-xen, decrement refcount)
(lttd-xen : writes the last subbuffers. decrement refcount, ressources are
freed)

If no lttd-xen is mapping the buffers, we simply free then upon
lttctl-xen -r.

> I think it's hard to
> reliably provide process scope for physical resources without assistance
> from the guest kernel (but I'll be happy to be proven wrong!).
> 

If we can associate a file descriptor with the ressource, then it
becomes much easier to associate. Aren't there already hypercalls like
"get event channel descriptor" which allocates a file descriptor ? I
wonder if we could use a similar mechanism to identify the use of a Xen
ressource by a process within dom0.

> In xentrace (which incidentally is this intended to replace? Or complement?)

My goal is currently to show if we can achieve performance improvement
over xentrace by using the lttng tracer. More practically, I want to use
data gathered from Xen, dom0 and domUs in the same analysis tool to perform
system-wide analysis

> the buffer lifetime is quite separate from the lifetime of xentrace daemon
> invocations. That's just what turned out to be easiest to implement with no
> guest kernel modifications. However if that's not a good model for LTTng
> then we can certainly consider if extra support, in guest kernel or in
> hypervisor, is warranted.
> 

As explained in this message, our model separates the lifetime of the
buffers from the daemon in some way : they stay allocated as long as
either Xen or the lttd daemon keeps a refcount incremented on them.

One of the benefits to use the scheme we provide is that we can easily
perform all the actions done by lttctl-xen directly from within the Xen
hypervisor : this is useful from stopping the tracing quickly when some
condition is detected.

Mathieu

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel