Design session notes: Reducing the number of pages always mapped in Xen: Next steps

[Marek Marczykowski-Górecki <marmarek@xxxxxxxxxxxxxxxxxxxxxx>]

Hi, 

Here is what I managed to capture, unfortunately some parts slipped by
me.

Julien: remove directmap, avoid speculative reading it
  series sent 6 months ago - enough to remove directmap, but not perfect
  resolving virtual adresses requires mapping/unmapping page tables - 
significant perf hit
Andrew: a lot of attacks read direct from directmap
  end goal "address space isolation" - no sensitive info mapped into xen
  directmap is not only place, but the large part of the problem
  single heap semi-rely on directmap
Bertrand: you don't know heap size for a guest up front, maybe can be specified 
manually?
Jan: why would that help?
Bertrand: no need to extend heap at guest runtime
Andrew: reduce number of translation (virt <-> phys) by using better data 
structures
  even getting address space isolation just for HVM guests will be huge 
improvement
Jan: zap the directmap when switching to HVM, and reinstantiate when switching 
to PV?
Andrew: just not have it mapped; some things needs to be mapped, like vcpu
  structure but not sensitive info, on fast path skip speculative mitigations,
  but when hitting slow path (page fault), apply speculative migitations and 
restore directmap;
  this makes fast path faster and slow path slower
Roger: what about auto eIBRS?
Andrew: it helps only in newer hardware, there is still older hardware
  even with retblead, the fast path with address space isolation would remain 
fast;
  it's also about future-proofing, many new bugs will not require HV changes
George: the slow path would still require adjustments/mitigations, likely
Julien: map specific pages individually, not whole directmap, keep common 
xenheap mapped
Andrew: address space isolation helps also with non-speculative attacks, and 
also per-guest heaps would further isolate sensitive data
Julien: the problem with page found approach is finding all the places and data 
that is safe and needed for fast paths
Andrew: implement faulting and then profile, then see whether common hits are 
safe to keep mapped, but if not try to rearrange algorithms/data structures
Bertrand: adjust how Xen is linked, isolate fast path areas from slow path 
areas to be able to switch them on/off fast
Andrew: struct vcpu and struct domain is a dumping ground for everything, some 
parts will need moving too
  for example: register data for own vcpus - probably safe, but for different 
vcpu of the same guest probably not safe, vcpu for different guest definitely 
not
Bertrand: risk moving the problem somewhere else? the problem of defining what 
is safe
Andrew: you can identify when it's in the fast path
Jan: besides registers and guest own memory, is there anything else secret?
Andrew: we have more luck than Linux, because for example Xen has no in-Xen 
crypto libraries;
  but also, for example you can figure code paths by looking at stacks
  not much more secret data
Bertrand: if we try to unmap guest-specific data (Jan's idea), don't we solve 
the problem more efficient way?
Andrew: it's risky
  per-vcpu mapping is easy for HVM, but not for PV, because top level page
  table is chosen by the PV kernel, and Linux does sometimes run multiple vcpus
  using the same page tables -> no per-vcpu mapping
George: close to time limit, lets go to conclusions
Julien: figure out next steps, what to do with the series from 6 months ago
  remove directmap
  make virt<->mfn mapping easy to use
Jan: this feels like going too far, if we only need to remove few secret data
Julien: directmap is about whole guest memory
Daniel Smith(?): what is the overlap with SEV
Andrew: doesn't really overlap with encrypted VMs
  both Intel and AMD encrypted VMs assume hypervisor may have a mapping of 
encrypted pages
  if directmap is present, you have still cache timing attacks, removing 
directmap helps with that
Bertrand: also benefits from safety POV - limits the scope for evaluation
Andrew: accidental out of bounds write will be a page fault - easier to notice
Jan: on demand mapping of xenheap, that means 4k mappings of everything; can we 
do better, to preserve superpages?
Julien: few other structures to consider
Andew: EPT page tables are not sensitive, MSR permissions also not, because 
guest vcpu can recover them anyway
Jan: that data actually can be sensitive, but you can't do anything about it

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

Attachment: signature.asc
Description: PGP signature