Re: [Xen-devel] RFC v1: Xen block protocol overhaul - problem statement (with pictures!)

[Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>]

On Thu, Jan 24, 2013 at 10:06:56AM +0000, Ian Campbell wrote:
> On Wed, 2013-01-23 at 16:59 +0000, Konrad Rzeszutek Wilk wrote:
> > On Wed, Jan 23, 2013 at 03:41:17PM +0000, Ian Campbell wrote:
> > > On Wed, 2013-01-23 at 15:21 +0000, Konrad Rzeszutek Wilk wrote:
> > > > > Have you consider variable length requests? This would let the request
> > > > > size scale with the number of segments required for that request, and
> > > > > allow you to cache align the ends of the requests without wasting the
> > > > > extra space that including the worst case number of segments would
> > > > > imply. e.g. a small write would take 32-bytes (padded to 64 if you 
> > > > > must)
> > > > > and a larger one would take 196 (padded to 256). You should end up 
> > > > > with
> > > > > more efficient use of the space in the ring this way.
> > > > 
> > > > Yes. If I recall right, the D) had it as "we could negotiate a proper 
> > > > alignment for
> > > > each request/response structure.". The idea is pretty much the same as 
> > > > yours
> > > > - pad to the cacheline if the cacheline is bigger than the request or 
> > > > response.
> > > 
> > > It would be interesting to know whether it is cache line bouncing or
> > > smaller ring capacity (which padding reduces) which becomes the
> > > bottleneck.
> > > 
> > > My money is, unhelpfully, on "both, depending on workload".
> > 
> > and on the guest pCPU distribution potentially. The one workload I've been
> > doing is to stick guests on different sockets and also turning
> > hardware prefetching off (and also Turbo Mode for good measure).
> 
> This last two are constructing a rather artificial scenario though
> aren't they?

Yes. But they give a great insight on the cache issues.
> 
> > This way it is the worst of the worlds - but it can provide me with the
> > data whether this cache issue is a problem or not.
> > 
> > > 
> > > 
> > > 
> > > _______________________________________________
> > > Xen-devel mailing list
> > > Xen-devel@xxxxxxxxxxxxx
> > > http://lists.xen.org/xen-devel
> > > 
> 
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel