[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-devel] [PATCH 1/3] x86: avoid flush IPI when possible
>>> On 10.02.16 at 18:51, <andrew.cooper3@xxxxxxxxxx> wrote: > On 10/02/16 15:37, Jan Beulich wrote: >>>>> On 10.02.16 at 16:00, <andrew.cooper3@xxxxxxxxxx> wrote: >>> On 10/02/16 12:56, Jan Beulich wrote: >>>> Since CLFLUSH, other than WBINVD, is a coherency domain wide flush, >>> I can't parse this sentence. >> Should have been "..., is a cache coherency domain wide flush, ..." - >> does it read any better then? > > I believe, given the code in the patch, your intent is "if we WBINVD, we > don't need to IPI other cores cache flushing reasons". I don't see how this can be read from the sentence. The primary statement is makes if "CLFLUSH is a cache coherency domain wide flush". A secondary statement is that this is different from WBINVD. > However, given your comment below... > >> >>> CLFUSH states "Invalidates from every level of the cache hierarchy in >>> the cache coherence domain" >>> >>> WBINVD however states "The instruction then issues a special-function >>> bus cycle that directs external caches to also write back modified data >>> and another bus cycle to indicate that the external caches should be >>> invalidated." >>> >>> I think we need input from Intel and AMD here as to the behaviour and >>> terminology here, and in particular, where the coherency domain >>> boundaries are. All CPUs, even across multiple sockets, see coherent >>> caching, but it is unclear whether this qualifies them to be in the same >>> cache coherency domain per the instruction spec. >> Linux already doing what this patch switches us to, I'm not sure >> we need much extra input. >> >>> In particular, given the architecture of 8-socket systems and 45MB of >>> RAM in L3 caches, does wbinvd seriously drain all caches everywhere? >> Not everywhere, just on the local socket (assuming there's no external >> cache). > > If this is true, then it is clearly not safe to omit the IPIs. When using CLFLUSH it is safe, while when using WBINVD it's not. >>> Causing 45MB of data to move to remote memory controllers all at once >>> would cause a massive system stall. >> That's why it takes (as we know) so long. See the figure in SDM Vol 3 >> section "Invalidating Caches and TLBs". > > I presume you mean Figure 2-10. WBINVD Invalidation of Shared and > Non-Shared Cache Hierarchy? > > This quite clearly shows that WBINVD will not invalidate or write back > the L1 caches for other cores in the same processor. > > Have I misunderstood the logic for choosing when to omit the IPIs? I'm afraid you did, or else I must have introduced a (latent, because I didn't notice any issues so far) bug. Jan _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxx http://lists.xen.org/xen-devel
|
Lists.xenproject.org is hosted with RackSpace, monitoring our |