LBR and Sapphire Rapids

[Andrew Cooper <Andrew.Cooper3@xxxxxxxxxx>]

Hello,

Testing has identified that VMs on SPR still crash when trying to turn
on LBR, and this is imminently going to cease being a "future" problem.

There is a series out about this, but there is some general confusion
creating mistakes, so I want to try and lay things out coherently here
in one go.

Right now (for Intel), the PDCM CPUID bit is hidden, HVM will #GP for
reads, while PV blindly returns 0.

The first time a vCPU tries to enable MSR_DBG_CTRL.LBR, we either set up
the MSR load/save lists for the LBR MSRs, and de-intercept them, or we
crash the VM if we can't figure out what to do.

LBR MSRs are never preserved on migrate.  A VM that is migrated will (at
best) only see corruption of its data.  If it migrates between otherwise
identical systems that have differing hyperthread settings, it may find
that the LBR stack is a different size.  If it migrates to a system with
a different LBR format, then pretty much everything will explode.


Longterm, we want to support Arch LBR, but we're a long way off being
able to do that.  In the meantime, we need to make VMs not crash. 
IceLake (server at least, not sure about client) has both Arch LBR and
model specific LBR.  Sapphire Rapids does not have model specific LBR.

Also, we cannot advertise the PCDM bit until we've got MSRs properly
accounted for in the migration safety checks (which is still a work in
progress).

From a "not crashing on migrate" point of view, migration need to be
blocked in any case where the LBR format changes (and other cases too). 
Which also means that by default, VMs want to be told "no model-specific
LBR".  But for backwards compatibility we also need a way for the user
to say "please let it still use model specific LBR", and this can't be
an architectural CPUID bit  (But I think it can be expressed as a
combination of PDCM=1,format!=0x3f,ARCH_LBR=0)


But it still doesn't help with SPR today.

On SPR, MSR_DBG_CTRL.LBR is a write-discard bit.  There really are no
model specific LBRs, so we should emulate it as write discard too.  More
generally, I think we should apply that to any system were we don't know
the model-specific indices.

I think this will be sufficient to avoid crashing guests on SPR.  Any
software actually expecting to use model specific LBR would need a model
table anyway just like Xen has, and will not get it updated with SPR's
model number, so for the (more) common case of not having migrated,
things should turn off cleanly.

~Andrew