Re: [Xen-devel] [Hackathon minutes] PV block improvements

[Wei Liu <wei.liu2@xxxxxxxxxx>]

On Fri, Jun 21, 2013 at 04:16:25PM -0400, Konrad Rzeszutek Wilk wrote:
> On Fri, Jun 21, 2013 at 07:10:59PM +0200, Roger Pau Monné wrote:
> > Hello,
> > 
> > While working on further block improvements I've found an issue with
> > persistent grants in blkfront.
> > 
> > Persistent grants basically allocate grants and then they are never
> > released, so both blkfront and blkback keep using the same memory pages
> > for all the transactions.
> > 
> > This is not a problem in blkback, because we can dynamically choose how
> > many grants we want to map. On the other hand, blkfront cannot remove
> > the access to those grants at any point, because blkfront doesn't know
> > if blkback has this grants mapped persistently or not.
> > 
> > So if for example we start expanding the number of segments in indirect
> > requests, to a value like 512 segments per requests, blkfront will
> > probably try to persistently map 512*32+512 = 16896 grants per device,
> > that's much more grants that the current default, which is 32*256 = 8192
> > (if using grant tables v2). This can cause serious problems to other
> > interfaces inside the DomU, since blkfront basically starts hoarding all
> > possible grants, leaving other interfaces completely locked.
> > 
> > I've been thinking about different ways to solve this, but so far I
> > haven't been able to found a nice solution:
> > 
> > 1. Limit the number of persistent grants a blkfront instance can use,
> > let's say that only the first X used grants will be persistently mapped
> > by both blkfront and blkback, and if more grants are needed the previous
> > map/unmap will be used.
> > 
> > 2. Switch to grant copy in blkback, and get rid of persistent grants (I
> > have not benchmarked this solution, but I'm quite sure it will involve a
> > performance regression, specially when scaling to a high number of domains).
> > 

Any chance that the speed of copying is fast enough for block devices?

> > 3. Increase the size of the grant_table or the size of a single grant
> > (from 4k to 2M) (this is from Stefano Stabellini).
> > 
> > 4. Introduce a new request type that we can use to request blkback to
> > unmap certain grefs so we can free them in blkfront.
> 
> 
> 5). Lift the limit of grant pages a domain can have.

If I'm not mistaken, this is basically the same as "increase the size of
the grant_table" in #3.

> 
> 6). Have an outstanding of grant pools that are mapped to a guest and
> recycle them? That way both netfront and blkfront could use them as needed?
> 

Is there an easy way to instrument the network stack to use those pages
only?


Wei.

> > 
> > So far none of them looks like a suitable solution.
> > 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxx
> http://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel