Re: [RFC PATCH 0/6] virtio: Solution to restrict memory access under Xen using xen-virtio DMA ops layer

[Oleksandr <olekstysh@xxxxxxxxx>]

On 15.04.22 11:44, Michael S. Tsirkin wrote:


Hello Michael



> On Thu, Apr 14, 2022 at 10:19:27PM +0300, Oleksandr Tyshchenko wrote:
> > From: Oleksandr Tyshchenko <oleksandr_tyshchenko@xxxxxxxx>
> >
> > Hello all.
> >
> > The purpose of this RFC patch series is to add support for restricting memory 
> > access under Xen using specific
> > grant table based DMA ops layer. Patch series is based on Juergen Gross’ 
> > initial work [1] which implies using
> > grant references instead of raw guest physical addresses (GPA) for the virtio 
> > communications (some kind of
> > the software IOMMU).
> >
> > The high level idea is to create new Xen’s grant table based DMA ops layer for 
> > the guest Linux whose main
> > purpose is to provide a special 64-bit DMA address which is formed by using the 
> > grant reference (for a page
> > to be shared with the backend) with offset and setting the highest address bit 
> > (this is for the backend to
> > be able to distinguish grant ref based DMA address from normal GPA). For this 
> > to work we need the ability
> > to allocate contiguous (consecutive) grant references for multi-page 
> > allocations. And the backend then needs
> > to offer VIRTIO_F_ACCESS_PLATFORM and VIRTIO_F_VERSION_1 feature bits (it must 
> > support virtio-mmio modern
> > transport for 64-bit addresses in the virtqueue).
> I'm not enough of a xen expert to review this, and I didn't get
> all patches, but I'm very happy to see that approach being
> taken. VIRTIO_F_ACCESS_PLATFORM and VIRTIO_F_VERSION_1 are
> exactly the way to declare not all of memory is accessible.
> Thanks!
I am happy to hear that! Thank you.


Regarding the "all patches" I have already redirect missing ones, I hope 
you and Christoph will get them.
Sorry for the inconvenience.


> > Xen's grant mapping mechanism is the secure and safe solution to share pages 
> > between domains which proven
> > to work and works for years (in the context of traditional Xen PV drivers for 
> > example). So far, the foreign
> > mapping is used for the virtio backend to map and access guest memory. With the 
> > foreign mapping, the backend
> > is able to map arbitrary pages from the guest memory (or even from Dom0 
> > memory). And as the result, the malicious
> > backend which runs in a non-trusted domain can take advantage of this. Instead, 
> > with the grant mapping
> > the backend is only allowed to map pages which were explicitly granted by the 
> > guest before and nothing else.
> > According to the discussions in various mainline threads this solution would 
> > likely be welcome because it
> > perfectly fits in the security model Xen provides.
> >
> > What is more, the grant table based solution requires zero changes to the Xen 
> > hypervisor itself at least
> > with virtio-mmio and DT (in comparison, for example, with "foreign mapping + 
> > virtio-iommu" solution which would
> > require the whole new complex emulator in hypervisor in addition to new 
> > functionality/hypercall to pass IOVA
> > from the virtio backend running elsewhere to the hypervisor and translate it to 
> > the GPA before mapping into
> > P2M or denying the foreign mapping request if no corresponding IOVA-GPA mapping 
> > present in the IOMMU page table
> > for that particular device). We only need to update toolstack to insert a new 
> > "xen,dev-domid" property to
> > the virtio-mmio device node when creating a guest device-tree (this is an 
> > indicator for the guest to use grants
> > and the ID of Xen domain where the corresponding backend resides, it is used as 
> > an argument to the grant mapping
> > APIs). It worth mentioning that toolstack patch is based on non  upstreamed yet 
> > “Virtio support for toolstack
> > on Arm” series which is on review now [2].
> >
> > Please note the following:
> > - Patch series only covers Arm and virtio-mmio (device-tree) for now. To enable 
> > the restricted memory access
> >    feature on Arm the following options should be set:
> >    CONFIG_XEN_VIRTIO = y
> >    CONFIG_XEN_HVM_VIRTIO_GRANT = y
> > - Some callbacks in xen-virtio DMA ops layer (map_sg/unmap_sg, etc) are not 
> > implemented yet as they are not
> >    needed/used in the first prototype
> >
> > Patch series is rebased on Linux 5.18-rc2 tag and tested on Renesas Salvator-X 
> > board + H3 ES3.0 SoC (Arm64)
> > with standalone userspace (non-Qemu) virtio-mmio based virtio-disk backend 
> > running in Driver domain and Linux
> > guest running on existing virtio-blk driver (frontend). No issues were 
> > observed. Guest domain 'reboot/destroy'
> > use-cases work properly. I have also tested other use-cases such as assigning 
> > several virtio block devices
> > or a mix of virtio and Xen PV block devices to the guest.
> >
> > 1. Xen changes located at (last patch):
> > https://github.com/otyshchenko1/xen/commits/libxl_virtio_next
> > 2. Linux changes located at:
> > https://github.com/otyshchenko1/linux/commits/virtio_grant5
> > 3. virtio-disk changes located at:
> > https://github.com/otyshchenko1/virtio-disk/commits/virtio_grant
> >
> > Any feedback/help would be highly appreciated.
> >
> > [1] https://www.youtube.com/watch?v=IrlEdaIUDPk
> > [2] 
> > https://lore.kernel.org/xen-devel/1649442065-8332-1-git-send-email-olekstysh@xxxxxxxxx/
> >
> > Juergen Gross (2):
> >    xen/grants: support allocating consecutive grants
> >    virtio: add option to restrict memory access under Xen
> >
> > Oleksandr Tyshchenko (4):
> >    dt-bindings: xen: Add xen,dev-domid property description for
> >      xen-virtio layer
> >    virtio: Various updates to xen-virtio DMA ops layer
> >    arm/xen: Introduce xen_setup_dma_ops()
> >    arm/xen: Assign xen-virtio DMA ops for virtio devices in Xen guests
> >
> >   .../devicetree/bindings/virtio/xen,dev-domid.yaml  |  39 +++
> >   arch/arm/include/asm/xen/xen-ops.h                 |   1 +
> >   arch/arm/mm/dma-mapping.c                          |   5 +-
> >   arch/arm/xen/enlighten.c                           |  11 +
> >   arch/arm64/include/asm/xen/xen-ops.h               |   1 +
> >   arch/arm64/mm/dma-mapping.c                        |   5 +-
> >   arch/x86/mm/init.c                                 |  15 +
> >   arch/x86/mm/mem_encrypt.c                          |   5 -
> >   arch/x86/xen/Kconfig                               |   9 +
> >   drivers/xen/Kconfig                                |  20 ++
> >   drivers/xen/Makefile                               |   1 +
> >   drivers/xen/grant-table.c                          | 238 +++++++++++++--
> >   drivers/xen/xen-virtio.c                           | 335 +++++++++++++++++++++
> >   include/xen/arm/xen-ops.h                          |  20 ++
> >   include/xen/grant_table.h                          |   4 +
> >   include/xen/xen-ops.h                              |  13 +
> >   16 files changed, 679 insertions(+), 43 deletions(-)
> >   create mode 100644 Documentation/devicetree/bindings/virtio/xen,dev-domid.yaml
> >   create mode 100644 arch/arm/include/asm/xen/xen-ops.h
> >   create mode 100644 arch/arm64/include/asm/xen/xen-ops.h
> >   create mode 100644 drivers/xen/xen-virtio.c
> >   create mode 100644 include/xen/arm/xen-ops.h
> >
> > --
> > 2.7.4
--
Regards,

Oleksandr Tyshchenko