Best way to use altp2m to support VMFUNC EPT-switching?

["Johnson, Ethan" <ejohns48@xxxxxxxxxxxxxxxx>]

Hi all,

I'm looking for some pointers on how Xen's altp2m system works and how it's 
meant to be used with Intel's VMFUNC EPT-switching for secure isolation within 
an HVM/PVH guest's kernelspace.

Specifically, I am attempting to modify Xen to create (on request by an 
already-booted, cooperative guest with a duly modified Linux kernel) a second 
set of extended page tables that have access to additional privileged regions 
of host-physical memory (specifically, a page or two to store some sensitive 
data that we don't want the guest kernel to be able to overwrite, plus some 
host-physical MMIO ranges, specifically the xAPIC region). The idea is that the 
guest kernel will use VMFUNC to switch to the alternate EPTs and call "secure 
functions" provided (by the hypervisor) as read-only code to be executed in 
non-root mode on the alternate EPT, allowing certain VM-exit scenarios (namely, 
sending an IPI to another vCPU of the same domain) to be handled without 
exiting non-root mode. Hence, these extra privileged pages should only be 
visible to the alternative p2m that the "secure realm" functions live in. 
(Transitions between the secure- and insecure-realm EPTs will be through 
special read-only "trampoline" code pages that ensure the untrusted guest 
kernel can only enter the secure realm at designated entry points.)

Looking at Xen's existing altp2m code, I get the sense that Xen is already 
designed to support something at least vaguely like this. I have not, however, 
been able to find much in the way of documentation on altp2m, so I am reaching 
out to see if anyone can offer pointers on how to best use it.

What is the intended workflow (either in the toolstack or within the hypervisor 
itself) for creating and configuring an altp2m that should have access to 
additional host-physical frames that are not present in the guest's main p2m?

FWIW, once the altp2m has been set up in this fashion, we don't anticipate 
needing to fiddle with its mappings any further as long as the guest is running 
(so I'm thinking *maybe* the "external" altp2m mode will suffice for this). In 
fact, we may not even need to have any "overlap" between the primary and 
alternative p2m except the trampoline pages themselves (although this aspect of 
our design is still somewhat in flux).

I've noticed a function, do_altp2m_op(), in the hypervisor 
(xen/arch/x86/hvm/hvm.c) that seems to implement a number of altp2m-related 
hypercalls intended to be called from the dom0. Do these hypercalls already 
provide a straightforward way to achieve my goals described above entirely via 
(a potentially modified version of) the dom0 toolstack? Or would I be better 
off creating and configuring the altp2m from within the hypervisor itself, 
since I want to map low-level stuff like xAPIC MMIO ranges into the altp2m?

Thank you in advance for your time and assistance!

Sincerely,

Ethan Johnson
Computer Science PhD candidate, Systems group, University of Rochester
mailto:ejohns48@xxxxxxxxxxxxxxxx