[Minios-devel] [UNIKRAFT PATCH 0/6] Save extended registers on context switch

[Florian Schmidt <florian.schmidt@xxxxxxxxx>]

Unikraft supports compiling code with support for extended registers.
However, there is no logic in place to save and restore those registers when
switching contexts between threads. This means that multiple threads using
XMM registers will conflict.

This patch series introduces functionality to save and restore those
extended register sets for SSE (XMM) and AVX (YMM) registers. Support for
ZMM (AVX-512) registers is theoretically there, but not enabled during the
boot code, and for lack of a testing machine not currently tested.


Some remarks:

This patch series moves initialization of FP/SSE/AVX into the entry64.S
early boot code. THe way unikraft is set up, all C code is compiled with the
same flags, and even disabling all those extended command sets for setup.c
doesn't solve the problem, because the debug printing routines might use
VMOVAPS, for example. Thus, it is safer to do the enabling in assembly and
not risk #UD faults.

This patch series only enables support for x86. I remember a discussion
during the first large Arm patch series about using more than just the
generic registers for Arm. Can one of the people with deep knowledge about
the architecture comment how complicated it would be to do something similar
for Arm?
Also, the patch series is a little rough around the edges with regard to
architecture-agnostic support sw_ctx.h and sw_ctx.c. However, since there is
no threading support for Arm yet, these files aren't used by Arm at all at
the moment, and revisiting them at that point shouldn't be too hard.

Finally, I also invested some time into investigating a lazy swiching
routine, with threads only starting to save their extended register context
once they first use instructions from the extended instruction sets. While
lazy switching is not very popular any more, I figured in a unikernel, it
might still be useful, especially since we don't have to worry about
information leaking, which is one of the issues with it on general-purpose
OSs.
However, this requires switching off SSE/AVX/etc. when switching to a fresh
thread, so that the #UD fault can be trapped to find out when a thread
started using exended instructions, and potentially back and forth
on every thread context switch. Enabling and disabling these options
requires writing to CR0 and CR4, which is excruciatingly slow on KVM
compared to an XSAVE (by about a factor 20 on my test machine), and while
the difference between the two isn't quite as bad on Xen-PV, it's still not
great. I shelved this for now and decided to go with eating the XSAVE
overhead on every switch instead, which also makes for much more compact
logic.


Florian Schmidt (6):
  plat: check for and enable extended CPU features
  plat: Add -DxxxPLAT define for each platform
  plat/common: add include guards to include/x86/cpu.h
  plat: Add global struct to keep x86 CPU information
  plat/common: Add functionality to save and restore extended (x86) registers
  plat/common: Add a notice regarding trap handling

 plat/common/include/sw_ctx.h       |   9 ++-
 plat/common/include/x86/cpu.h      | 118 +++++++++++++++++++++++++++--
 plat/common/include/x86/cpu_defs.h |  22 ++++++
 plat/common/sw_ctx.c               |  14 +++-
 plat/common/x86/cpu_features.c     |  37 +++++++++
 plat/common/x86/traps.c            |  12 +++
 plat/kvm/Makefile.uk               |   4 +
 plat/kvm/x86/entry64.S             |  54 +++++++++++--
 plat/kvm/x86/setup.c               |  17 +----
 plat/linuxu/Makefile.uk            |   4 +
 plat/linuxu/setup.c                |   7 ++
 plat/xen/Makefile.uk               |   7 +-
 plat/xen/x86/entry64.S             |  63 +++++++++++++--
 plat/xen/x86/setup.c               |  15 +---
 14 files changed, 328 insertions(+), 55 deletions(-)
 create mode 100644 plat/common/x86/cpu_features.c

-- 
2.19.1


_______________________________________________
Minios-devel mailing list
Minios-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/minios-devel