Re: Armv8A - use of Xen and hypervisor EL2 translation tables to extend bridge address span

[Bertrand Marquis <Bertrand.Marquis@xxxxxxx>]

Hi Chris,

> On 17 Jan 2021, at 16:15, Kris Chaplin <kris.chaplin@xxxxxxxxxxxxxxx> wrote:
> 
> Hello,
> 
> I have a very specific challenge that I'm looking to solve, and I think that 
> a Hypervisor such as Xen, potentially with some modifications may help me get 
> there.
> 
> In short, I have a ARM 64-bit v8A device (Cortex-A53) that has a bridge to 
> custom logic with a limited address span, and I'm looking to address more 
> range than this brige has capacity for in a way that is transparent to the 
> operating system.  In the hardware, I have a windowed controller that I can 
> control at run-time to expose a subset of the address space at any one time, 
> but never the full range.
> 
> My thoughts are that in the OS, I could create a device tree to instantiate a 
> bridge to the address space that has range for the full capacity I'm looking 
> for at an unused portion of the IPA address map, and the use the hypervisor 
> to trap page faults to VTTBR0_EL2-mapped pages, get the hypervisor to steer 
> the windowed controller (if it is not already pointing to the correct window 
> being addressed) and then clear the page fault, and allow the OS to continue. 
>  In this way, the OS would be none-the-wiser to the existance of the windowed 
> controller, and software/drivers mapping to this bridge would use the 
> interface transparently (albeit with a performance hit).

>From the theory this is possible for sure.

> 
> Perhaps I've not explained that too well.  It's quite a specific challenge, 
> that I would assume would require some customization of Xen.  Has anything 
> similar to this been done before, or can anyone point me in the direction of 
> if this is feasible, and where to research next?

You will need to implement a driver in Xen to do that and catch some accesses 
and play with mappings inside Xen.
There is not something doing exactly that in Xen but you could look at what the 
vPCI driver for Arm under development right now is doing:
https://gitlab.com/xen-project/fusa/xen-integration/-/tree/integration/pci-passthrough

In this the vpci is catching accesses from guest to the pci configuration space 
and is emulating them.

You might find some other examples of parts of Xen doing things like that in 
other places (like the GIC driver emulating accesses to part of the interrupt 
controller).

Hope that can help you start even though it does not fully do exactly the stuff 
you want.

Cheers
Bertrand

> 
> Appreciate the help.
> 
> regards
> 
> Kris
> 
>