Re: [Xen-devel] [Hackathon minutes] PV block improvements

[Matt Wilson <msw@xxxxxxxxxx>]

On Fri, Jun 21, 2013 at 07:10:59PM +0200, Roger Pau Monné wrote:
> Hello,
> 
> While working on further block improvements I've found an issue with
> persistent grants in blkfront.
> 
> Persistent grants basically allocate grants and then they are never
> released, so both blkfront and blkback keep using the same memory pages
> for all the transactions.
> 
> This is not a problem in blkback, because we can dynamically choose how
> many grants we want to map. On the other hand, blkfront cannot remove
> the access to those grants at any point, because blkfront doesn't know
> if blkback has this grants mapped persistently or not.
> 
> So if for example we start expanding the number of segments in indirect
> requests, to a value like 512 segments per requests, blkfront will
> probably try to persistently map 512*32+512 = 16896 grants per device,
> that's much more grants that the current default, which is 32*256 = 8192
> (if using grant tables v2). This can cause serious problems to other
> interfaces inside the DomU, since blkfront basically starts hoarding all
> possible grants, leaving other interfaces completely locked.

Yikes.

> I've been thinking about different ways to solve this, but so far I
> haven't been able to found a nice solution:
> 
> 1. Limit the number of persistent grants a blkfront instance can use,
> let's say that only the first X used grants will be persistently mapped
> by both blkfront and blkback, and if more grants are needed the previous
> map/unmap will be used.

I'm not thrilled with this option. It would likely introduce some
significant performance variability, wouldn't it?

> 2. Switch to grant copy in blkback, and get rid of persistent grants (I
> have not benchmarked this solution, but I'm quite sure it will involve a
> performance regression, specially when scaling to a high number of domains).

Why do you think so?

> 3. Increase the size of the grant_table or the size of a single grant
> (from 4k to 2M) (this is from Stefano Stabellini).

Seems like a bit of a bigger hammer approach.

> 4. Introduce a new request type that we can use to request blkback to
> unmap certain grefs so we can free them in blkfront.

Sounds complex.

> So far none of them looks like a suitable solution.

I agree. Of these, I think #2 is worth a little more attention.

--msw

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel