Re: Is it time to start implementing Xen bindings for rust-vmm?

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Tue, 14 Sep 2021, Andrew Cooper wrote:
> On 14/09/2021 15:44, Alex Bennée wrote:
> > Andrew Cooper <andrew.cooper3@xxxxxxxxxx> writes:
> >
> >> On 13/09/2021 13:44, Alex Bennée wrote:
> >>> Hi,
> >>>
> >>> As we consider the next cycle for Project Stratos I would like to make
> >>> some more progress on hypervisor agnosticism for our virtio backends.
> >>> While we have implemented a number of virtio vhost-user backends using C
> >>> we've rapidly switched to using rust-vmm based ones for virtio-i2c,
> >>> virtio-rng and virtio-gpio. Given the interest in Rust for implementing
> >>> backends does it make sense to do some enabling work in rust-vmm to
> >>> support Xen?
> >>>
> >>> There are two chunks of work I can think of:
> >>>
> >>>   1. Enough of libxl/hypervisor interface to implement an IOREQ end point.
> >> No libxl here at all.
> >>
> >> As of Xen 4.15, there are enough stable interfaces to implement simple
> >> IOERQ servers.
> >>
> >> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fxapi-project%2Fvarstored%2Fcommit%2Ffde707c59f7a189e1d4e97c1a4ee1a2d0c378ad1&amp;data=04%7C01%7CAndrew.Cooper3%40citrix.com%7C08a3fe14704a4d6888cf08d9778ee5b2%7C335836de42ef43a2b145348c2ee9ca5b%7C0%7C0%7C637672277905441489%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=%2B1pKhIuqzCGkgYD4snd6jnxjoEJzrCgUdol%2FfA2kwOk%3D&amp;reserved=0
> >> was the commit where I removed the final unstable interface from
> >> varstored (terrible name) which is a dom0 backend for UEFI secure
> >> variable handling.  As such, it also serves as a (not totally simple)
> >> reference of an IOERQ server.
> >>
> >>
> >> There are a few bits and pieces of rust going on within Xen, and a whole
> >> load of plans.  Also, there is a lot of interest from other downstreams
> >> in being able to write Rust backends.
> >>
> >> We've got a placeholder xen and xen-sys crates, and placeholder work for
> >> supporting cross-compile as x86 PV and PVH stubdomains.
> > Are these in the rust-vmm project is elsewhere?
> 
> https://crates.io/crates/xen-sys
> 
> When I say placeholder, I really do mean placeholder.
> 
> To start this work meaningfully, we'd want to make a repo (or several)
> in the xen-project organisation on github or gitlab (we have both, for
> reasons), and set these as the upstream of the xen and xen-sys crates.
> 
> >> The want to have a simple IOREQ server compiled either as a dom0
> >> backend, or as a PV or PVH stubdomains influences some of the design
> >> decisions early on, but they're all no-brainers for the longevity of the
> >> work.
> > Just to clarify nomenclature is a PVH stubdomain what I'm referring to
> > as a bare metal backend, i.e: a unikernel or RTOS image that implements
> > the backend without having to transition between some sort of userspace
> > and it's supporting kernel?
> 
> I think so, yes, although calling it "bare metal" seems misleading for
> something which is a VM targetted at a specific hypervisor...
> 
> 
> >> I started work on trying to reimplement varstored entirely in Rust as a
> >> hackathon project, although ran out of time trying to make hypercall
> >> buffers work (there is a bug with Box and non-global allocators causing
> >> rustc to hit an assert().  In the short term, we'll have to implement
> >> hypercall buffers in a slightly more irritating way).
> >>
> >> Furthermore, stick to the stable hypercalls only.  Xen's C libraries are
> >> disaster for cross-version compatibility, and you absolutely do not want
> >> to recompile your rust program just to run it against a different
> >> version of the hypervisor.  The plan is to start with simple IOREQ
> >> servers, which are on fully stable interfaces, then stabilise further
> >> hypercalls as necessary to expand functionality.
> > Are the hypercalls mediated by a kernel layer or are you making direct
> > HVC calls (on ARM) to the hypervisor from userspace?
> 
> For a dom0 backends irrespective of architecture, you need to issue
> ioctl()'s on the appropriate kernel device.
> 
> For a PV/PVH stubdom, you should make a call into the hypercall_page
> https://xenbits.xen.org/docs/latest/guest-guide/x86/hypercall-abi.html
> because Intel and AMD used different instructions for their equivalent
> of HVC.
> 
> ARM doesn't have the hypercall page ABI, so I'd expect the hypercall
> implementation to expand to HVC directly.

See for example arch/arm64/xen/hypercall.S in Linux