Re: [Xen-devel] [PATCH] x86/flush: use APIC ALLBUT destination shorthand when possible

[Roger Pau Monné <roger.pau@xxxxxxxxxx>]

On Fri, Dec 27, 2019 at 03:03:57PM +0000, Andrew Cooper wrote:
> On 24/12/2019 12:44, Roger Pau Monne wrote:
> > If the flush mask matches the mask of online CPUs use the APIC ALLBUT
> > destination shorthand in order to send an IPI to all CPUs on the
> > system except the current one. This can only be safely used when no
> > CPU hotplug or unplug operations are taking place, no offline CPUs or
> > those have been onlined and parked and finally when all CPUs in the
> > system have been accounted for (ie: the number of CPUs doesn't exceed
> > NR_CPUS and APIC IDs are below MAX_APICS).
> >
> > This is specially beneficial when using the PV shim, since using the
> > shorthand avoids performing an APIC register write (or multiple ones
> > if using xAPIC mode) for each destination in the flush mask.
> >
> > The lock time on a 32 vCPU guest using the shim without the shorthand
> > is:
> >
> > Global lock flush_lock: addr=ffff82d0804b21c0, lockval=f602f602, not locked
> >   lock:228455938(79406065573135), block:205908580(556416605761539)
> >
> > Average lock time: 347577ns
> >
> > While the same guest using the shorthand:
> >
> > Global lock flush_lock: addr=ffff82d0804b41c0, lockval=d9c4d9bc, cpu=12
> >   lock:1890775(416719148054), block:1663958(2500161282949)
> >
> > Average lock time: 220395ns
> >
> > Approximately a 1/3 improvement in the lock time.
> >
> > Signed-off-by: Roger Pau Monné <roger.pau@xxxxxxxxxx>
> 
> While these are good stats, I'm somewhat hesitant about hacking this in
> like this.  For one, it is a substantial amount of ad-hoc logic in
> flush_area_mask()
> 
> Shorthand safety really should be part of the apic driver, not part of
> the TLB logic.

Yes, I've coded it this way because there are already similar APIC
hooks (ie: send_IPI_self). I can merge the shorthand functionality
with send_IPI_mask if that's preferred.

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel