Re: [Xen-devel] Question about DMA on 1:1 mapping dom0 of arm64

[Ian Campbell <ian.campbell@xxxxxxxxxx>]

On Fri, 2015-04-17 at 17:13 +0100, Stefano Stabellini wrote:
> On Fri, 17 Apr 2015, Ian Campbell wrote:
> > On Fri, 2015-04-17 at 15:34 +0100, Stefano Stabellini wrote:
> > > > > If I set dom0_mem to a small value (e.g. 512M), which makes all 
> > > > > physical memory
> > > > > of dom0 below 4G, everything goes fine.
> > > > 
> > > > So you are getting allocated memory below 4G?
> > > > 
> > > > You message on IRC suggested you weren't, did you hack around this?
> > > > 
> > > > I think we have two options, either xen_swiotlb_init allocates pages
> > > > below 4GB (e.g. __GFP_DMA) or we do something to allow xen_swiotlb_fixup
> > > > to actually work even on a 1:1 dom0.
> > > 
> > > I don't think that making xen_swiotlb_fixup work on ARM is a good idea:
> > > it would break the 1:1.
> > 
> > This would actually work though, I think, because this is the swiotlb so
> > we definitely have the opportunity to return the actual DMA address
> > whenever we use this buffer and the device will use it in the right
> > places for sure.
> 
> The code is pretty complex as is -- I would rather avoid adding more
> complexity to it.  For example we would need to bring back a mechanism
> to track dma address -> pseudo-physical address mappings on arm, even
> though it would be far simpler of course.

There's no need for any of that, just initialise start_dma_addr with the
correct DMA addr when you do the fixup and the swiotlb code will already
take care of everything, won't it?.

That DMA addr would be given to us by Xen as we do the flip.

> Also I think it makes sense to use the swiotlb buffer for its original
> purpose.
> 
> If we could introduce a mechanism to get a lower than 4G buffer in dom0,
> but matching the 1:1, I think it would make the maintenance much easier
> on the linux side.
> 
> 
> > The swiotlb buffer can't ever get reused for anything else so we don't
> > even need to worry about undoing the damage later.
> > 
> > > > Although the first option seems preferable at first glance it has the
> > > > short coming that it requires dom0 to have some memory below 4GB, which
> > > > might not necessarily be the case.
> > > 
> > > I think we should arrange dom0 to get some memory under 4G to begin
> > > with, not necessarily all of it.
> > 
> > It's another option for sure, the question is how to decide how much,
> > and how to make it configurable etc.
>  
> Adding a memory region below 4G should be relatively easy, right?  Is it
> sizing it up the problem? FYI the default size of the swiotlb buffer in
> Linux is 64M, but it shoul be able to cope with less than that.

Picking a size might be tricky, you can't just say "Linux is 64M", what
if that 64M gets used for something else first.

Given a machine to 2GB of low RAM and 126GB of high RAM and a 27G dom0
which fraction do you put below 4GB?

But maybe it works to just say that dom0 gets as much lowmem as the host
memory map and our ability to allocate allows for. That's not entirely
trivial either though, in case Xen has (by chance) allocated any lowmem
for itself the bank bin packing stuff will get even more complex.
Likewise firmware reserved regions.

Ian.



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel