Re: [Xen-devel] [Hackathon minutes] PV block improvements

[Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>]

On Sat, 22 Jun 2013, Wei Liu wrote:
> On Fri, Jun 21, 2013 at 04:16:25PM -0400, Konrad Rzeszutek Wilk wrote:
> > On Fri, Jun 21, 2013 at 07:10:59PM +0200, Roger Pau Monnà wrote:
> > > Hello,
> > > 
> > > While working on further block improvements I've found an issue with
> > > persistent grants in blkfront.
> > > 
> > > Persistent grants basically allocate grants and then they are never
> > > released, so both blkfront and blkback keep using the same memory pages
> > > for all the transactions.
> > > 
> > > This is not a problem in blkback, because we can dynamically choose how
> > > many grants we want to map. On the other hand, blkfront cannot remove
> > > the access to those grants at any point, because blkfront doesn't know
> > > if blkback has this grants mapped persistently or not.
> > > 
> > > So if for example we start expanding the number of segments in indirect
> > > requests, to a value like 512 segments per requests, blkfront will
> > > probably try to persistently map 512*32+512 = 16896 grants per device,
> > > that's much more grants that the current default, which is 32*256 = 8192
> > > (if using grant tables v2). This can cause serious problems to other
> > > interfaces inside the DomU, since blkfront basically starts hoarding all
> > > possible grants, leaving other interfaces completely locked.
> > > 
> > > I've been thinking about different ways to solve this, but so far I
> > > haven't been able to found a nice solution:
> > > 
> > > 1. Limit the number of persistent grants a blkfront instance can use,
> > > let's say that only the first X used grants will be persistently mapped
> > > by both blkfront and blkback, and if more grants are needed the previous
> > > map/unmap will be used.
> > > 
> > > 2. Switch to grant copy in blkback, and get rid of persistent grants (I
> > > have not benchmarked this solution, but I'm quite sure it will involve a
> > > performance regression, specially when scaling to a high number of 
> > > domains).
> > > 
> 
> Any chance that the speed of copying is fast enough for block devices?
> 
> > > 3. Increase the size of the grant_table or the size of a single grant
> > > (from 4k to 2M) (this is from Stefano Stabellini).
> > > 
> > > 4. Introduce a new request type that we can use to request blkback to
> > > unmap certain grefs so we can free them in blkfront.
> > 
> > 
> > 5). Lift the limit of grant pages a domain can have.
> 
> If I'm not mistaken, this is basically the same as "increase the size of
> the grant_table" in #3.

Yes, that was one of the things I was suggesting, but it needs
investigating: I wouldn't want that increasing the number of grant
frames would reach a different scalability limit of the data structure.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel