[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-devel] [Hackathon minutes] PV block improvements
On Thu, 27 Jun 2013, Roger Pau Monnà wrote: > On 21/06/13 20:07, Matt Wilson wrote: > > On Fri, Jun 21, 2013 at 07:10:59PM +0200, Roger Pau Monnà wrote: > >> Hello, > >> > >> While working on further block improvements I've found an issue with > >> persistent grants in blkfront. > >> > >> Persistent grants basically allocate grants and then they are never > >> released, so both blkfront and blkback keep using the same memory pages > >> for all the transactions. > >> > >> This is not a problem in blkback, because we can dynamically choose how > >> many grants we want to map. On the other hand, blkfront cannot remove > >> the access to those grants at any point, because blkfront doesn't know > >> if blkback has this grants mapped persistently or not. > >> > >> So if for example we start expanding the number of segments in indirect > >> requests, to a value like 512 segments per requests, blkfront will > >> probably try to persistently map 512*32+512 = 16896 grants per device, > >> that's much more grants that the current default, which is 32*256 = 8192 > >> (if using grant tables v2). This can cause serious problems to other > >> interfaces inside the DomU, since blkfront basically starts hoarding all > >> possible grants, leaving other interfaces completely locked. > > > > Yikes. > > > >> I've been thinking about different ways to solve this, but so far I > >> haven't been able to found a nice solution: > >> > >> 1. Limit the number of persistent grants a blkfront instance can use, > >> let's say that only the first X used grants will be persistently mapped > >> by both blkfront and blkback, and if more grants are needed the previous > >> map/unmap will be used. > > > > I'm not thrilled with this option. It would likely introduce some > > significant performance variability, wouldn't it? > > > >> 2. Switch to grant copy in blkback, and get rid of persistent grants (I > >> have not benchmarked this solution, but I'm quite sure it will involve a > >> performance regression, specially when scaling to a high number of > >> domains). > > > > Why do you think so? > > I've hacked a prototype blkback using grant_copy instead of persistent > grants, and removed the persistent grants support in blkfront and indeed > the performance of grant_copy is lower than persistent grants, and it > seems to scale much worse. I've run several fio read/write benchmarks, > using 512 segments per request on a ramdisk, and the output is the > following: > > http://xenbits.xen.org/people/royger/grant_copy/ Very impressive. We should consider doing the same experiment with netfront/netback at some point. _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxx http://lists.xen.org/xen-devel
|
Lists.xenproject.org is hosted with RackSpace, monitoring our |