|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-devel] [Hackathon minutes] PV block improvements
On 21/06/13 20:07, Matt Wilson wrote: > On Fri, Jun 21, 2013 at 07:10:59PM +0200, Roger Pau Monné wrote: >> Hello, >> >> While working on further block improvements I've found an issue with >> persistent grants in blkfront. >> >> Persistent grants basically allocate grants and then they are never >> released, so both blkfront and blkback keep using the same memory pages >> for all the transactions. >> >> This is not a problem in blkback, because we can dynamically choose how >> many grants we want to map. On the other hand, blkfront cannot remove >> the access to those grants at any point, because blkfront doesn't know >> if blkback has this grants mapped persistently or not. >> >> So if for example we start expanding the number of segments in indirect >> requests, to a value like 512 segments per requests, blkfront will >> probably try to persistently map 512*32+512 = 16896 grants per device, >> that's much more grants that the current default, which is 32*256 = 8192 >> (if using grant tables v2). This can cause serious problems to other >> interfaces inside the DomU, since blkfront basically starts hoarding all >> possible grants, leaving other interfaces completely locked. > > Yikes. > >> I've been thinking about different ways to solve this, but so far I >> haven't been able to found a nice solution: >> >> 1. Limit the number of persistent grants a blkfront instance can use, >> let's say that only the first X used grants will be persistently mapped >> by both blkfront and blkback, and if more grants are needed the previous >> map/unmap will be used. > > I'm not thrilled with this option. It would likely introduce some > significant performance variability, wouldn't it? > >> 2. Switch to grant copy in blkback, and get rid of persistent grants (I >> have not benchmarked this solution, but I'm quite sure it will involve a >> performance regression, specially when scaling to a high number of domains). > > Why do you think so? I've hacked a prototype blkback using grant_copy instead of persistent grants, and removed the persistent grants support in blkfront and indeed the performance of grant_copy is lower than persistent grants, and it seems to scale much worse. I've run several fio read/write benchmarks, using 512 segments per request on a ramdisk, and the output is the following: http://xenbits.xen.org/people/royger/grant_copy/ Roger. _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxx http://lists.xen.org/xen-devel
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |