Re: [Xen-devel] [PATCH 1/3] x86: avoid flush IPI when possible

["Jan Beulich" <JBeulich@xxxxxxxx>]

>>> On 10.02.16 at 16:00, <andrew.cooper3@xxxxxxxxxx> wrote:
> On 10/02/16 12:56, Jan Beulich wrote:
>> Since CLFLUSH, other than WBINVD, is a coherency domain wide flush,
> 
> I can't parse this sentence.

Should have been "..., is a cache coherency domain wide flush, ..." -
does it read any better then?

> CLFUSH states "Invalidates from every level of the cache hierarchy in
> the cache coherence domain"
> 
> WBINVD however states "The instruction then issues a special-function
> bus cycle that directs external caches to also write back modified data
> and another bus cycle to indicate that the external caches should be
> invalidated."
> 
> I think we need input from Intel and AMD here as to the behaviour and
> terminology here, and in particular, where the coherency domain
> boundaries are.  All CPUs, even across multiple sockets, see coherent
> caching, but it is unclear whether this qualifies them to be in the same
> cache coherency domain per the instruction spec.

Linux already doing what this patch switches us to, I'm not sure
we need much extra input.

> In particular, given the architecture of 8-socket systems and 45MB of
> RAM in L3 caches, does wbinvd seriously drain all caches everywhere? 

Not everywhere, just on the local socket (assuming there's no external
cache).

> Causing 45MB of data to move to remote memory controllers all at once
> would cause a massive system stall.

That's why it takes (as we know) so long. See the figure in SDM Vol 3
section "Invalidating Caches and TLBs".

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel