Re: [Xen-devel] Swiotlb

[Kieran Mansley <kmansley@xxxxxxxxxxxxxx>]

On Thu, 2007-04-05 at 16:10 +0100, Keir Fraser wrote:
> On 5/4/07 15:23, "Kieran Mansley" <kmansley@xxxxxxxxxxxxxx> wrote:
> > While writing a driver for a device doing lots of DMA I've hit an
> > "swiotlb_full()" problem.  This surprised me somewhat as I wouldn't have
> > expected to need the use of the software TLB - it's a 64 bit capable
> > device on a server with only 2 GB of RAM, and so I'd have expected to be
> > using a hardware TLB.  Is this a peculiarity of Xen, or should I be
> > right to be surprised?
> > 
> > I expect I can increase the size of the swiotlb to avoid this problem,
> > but thought I'd check that there isn't something more fundamentally
> > wrong first.
> 
> Well, I would agree that it sounds odd. You should definitely investigate --
> you don't want to be invoking the swiotlb on the data path for a modern
> high-speed device.

It's not quite on the data path - it's hitting this problem when
allocating buffers for network access, which typically will come from a
preallocated pool rather than being on demand, so perhaps it's not a big
problem.

I've had a quick look at what's going on.  The call into swiotlb comes
from dma_map_single() (which is I think the one defined in
arch/i386/kernel/pci-dma-xen.c).  This happens because the swiotlb
global is set to 1 in swiotlb_init().  It looks like this is unavoidable
as there's a comment there saying:
        /* Domain 0 always has a swiotlb. */

I could undefine CONFIG_SWIOTLB or set swiotlb_force to -1 and so avoid
the whole thing, but I'm not sure that would be a good thing - some
other drivers might depend on it.

I guess what I'm surprised about is that there's no check to see if
swiotlb is necessary on any of the above paths.  Perhaps I should be
calling something other than dma_map_single()?  Or perhaps this is just
the way it's supposed to be and I need to allocate a bigger swiotlb?
Any suggestions welcome.

Kieran


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel