[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] Hi,something about the xentrace tool

George Dunlap wrote:

There's no copying from the HV to the xentrace process.  But there is
copying from xentrace to the dom0 kernel for the output file.  Some
copying is necessary right now, because rather than writing out the
pages verbatim, xentrace writes out the pcpu before writing out each

void write_rec(unsigned int cpu, struct t_rec *rec, FILE *out)
   size_t written = 0;
   written += fwrite(&cpu, sizeof(cpu), 1, out);
   written += fwrite(rec, sizeof(*rec), 1, out);
   if ( written != 2 )
       PERROR("Failed to write trace record");

If we wanted to make it zero copy all the way from the HV to the disk,
we could have the xentrace process one stream per cpu, and do
whatever's necessary to use DMA.  (Does anyone know if O_DIRECT will
do direct DMA, or if one would have to use a raw disk?)

So you're saying if we didn't have to write the cpu number, then we could bypass stdio, and directly do a write() using the trace buffer? And this would be better because it would avoid a memory to memory copy, and use DMA immediately on the trace buffer memory? Do I understand you correctly? Assuming this is what you mean, allow me to correct a slight logic flaw. Stdio is there for a reason; Doing lots of raw I/O using very small buffers is highly inefficient. There's the overhead of kernel entry/exit and of setting up and tearing down DMA transactions. And writing to a block device will result in I/O's that are multiples of the devices' block size, so writing a 32 byte trace record will probably cause a 512-byte block to actually be written to disk. So bypassing stdio in this case will result in lots more disk accesses, lots more dma setup/teardown, and lots more system calls. In other words, the performance is going to horrible. The Stdio library greatly reduces all this overhead by buffering stuff in memory until there's enough to make a genuine I/O relatively efficient. In this case, the memory copies are intentional and beneficial; We do not want to eliminate them in our quest for "zero copy".


Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.