Re: Proposal for physical address based hypercalls

["Daniel P. Smith" <dpsmith@xxxxxxxxxxxxxxxxxxxx>]

On 9/29/22 04:16, Jan Beulich wrote:
> On 28.09.2022 15:32, dpsmith.dev wrote:
> > On 9/28/22 06:38, Jan Beulich wrote:
> > > For quite some time we've been talking about replacing the present virtual
> > > address based hypercall interface with one using physical addresses.  This is in
> > > particular a prerequisite to being able to support guests with encrypted
> > > memory, as for such guests we cannot perform the page table walks necessary to
> > > translate virtual to (guest-)physical addresses.  But using (guest) physical
> > > addresses is also expected to help performance of non-PV guests (i.e. all Arm
> > > ones plus HVM/PVH on x86), because of the no longer necessary address
> > > translation.
> > Greetings Jan,
> >
> > I think there are multiple issues in play here, but the two major ones
> > are 1.) eliminating the use of guest virtual addresses and 2.) handling
> > the change in the security model for hypercalls from encrypted VMs. As
> > Andy was pointing out, attempting to address (1) in a backwards
> > compatible approach will likely not arrive at a solution that can
> > address issue (2).
> It may not be sufficient, but it is (can be) a prereq.
As I stated below, it will start setting the precedent for using GPAs. 
The concern is two-fold, how much benefit can actually be achieved for 
an API/ABI that cannot be used in the final solution. And by focusing 
effort on an unusable API/ABI, how much will that reduce effort/focus on 
crafting an API/ABI that can be used.
> > IMHO, the only result from teaching the existing ABI
> > to speak GPAs instead of VAs will be to break current and new kernels of
> > the habit of using VAs. Beyond that I do not see how it will do anything
> > to prepare current OS kernels for running as encrypted VMs, at least for
> > AMD since that is the specification I have been focused on studying the
> > last couple of months.
> Plus we'd have code in the hypervisor then which deals with physical
> address based hypercall buffers. One less prereq to take care of for
> the (huge) rest of the work needed.
A question I would have is why not just RFC a GPA buffer helper 
framework for the hypervisor since it will get used by the new ABI and 
not spend effort retrofitting the current ABI. Some follow-on questions 
I would also ask is, moving forward would it be expected that new 
revisions of guests using the existing ABI would be expected to move to 
GPAs and how long do people see the existing ABI to continue in new 
guest revisions after the new ABI is adopted.
> > As for ABIv2, I understand and can appreciate Andy's desired approach.
> > Recently, especially with the hardware changes being introduced by SEV,
> > I would like to have considered a naive and more radical approach.
> > Currently hypercalls function using a more ioctl style. I would like to
> > suggest that a packet style interface similar to netlink be considered.
> > There are many benefits to adopting this type of interface that could be
> > covered in a larger RFC if there was any sense of willingness to
> > consider it. As a glimpse, a few benefits would be that arbitrary
> > buffers, continuations/asynchronous calls, and multi-call are all
> > natural consequence. It would also allow advanced extensions, such as an
> > optional PF_RING-like interface for zero-copy messaging from guest
> > user-space to hypervisor. While a packet interface could easily co-exist
> > with the existing ioctl-style interface, it would be a paradigm shift
> > from the past, though I feel ABIv2 was already going to be such a shift.
> > Anyway, just my 2¢.
> I'm sorry for my ignorance, but I have no knowledge of how netlink
> works.
Understood, and you are not the first. A very quick, and very loose, 
comparison is that currently hypercalls are managed as ioctl remote call 
with per version defined payload. This proposal would move to a packet 
dispatch where the packet is a free-form TLV that allows unknown 
elements/parameters to be present. This enables a newer toolstack , 
without requiring a constantly moving compatibility layer, to send a 
packet to an older hypervisor which can reject unknown elements and 
hypercalls to silently ignore unknown parameters. Similarly, an older 
toolstack will be able to send packets to a new hypervisor. And as I 
stated above, this approach naturally enables continuations/async 
operations and multi-call invocations. It is a significant departure, 
and thus would require substantial design and implementation work, but 
there is an opportunity here to do this work.
V/r,
DPS