Re: Keystone Issue

[CodeWiz2280 <codewiz2280@xxxxxxxxx>]

Is it possible to passthrough PCI devices to domU on the 32-bit arm
keystone?  Any info is appreciated.

I found some old information online that "gic-v2m" is required.  I'm
not sure if the GIC-400 on the K2E supports "gic_v2m".  Based on the
fact that there is no "gic-v2m-frame" tag in the K2E device tree i'm
guessing that it does not.

If it is possible, is there a good example for arm that I can follow?

On Wed, Jun 17, 2020 at 7:52 PM CodeWiz2280 <codewiz2280@xxxxxxxxx> wrote:
>
> On Wed, Jun 17, 2020 at 2:46 PM Stefano Stabellini
> <sstabellini@xxxxxxxxxx> wrote:
> >
> > On Wed, 17 Jun 2020, CodeWiz2280 wrote:
> > > On Tue, Jun 16, 2020 at 2:23 PM Marc Zyngier <maz@xxxxxxxxxx> wrote:
> > > >
> > > > On 2020-06-16 19:13, CodeWiz2280 wrote:
> > > > > On Tue, Jun 16, 2020 at 4:11 AM Marc Zyngier <maz@xxxxxxxxxx> wrote:
> > > > >>
> > > > >> On 2020-06-15 20:14, CodeWiz2280 wrote:
> > > > >>
> > > > >> [...]
> > > > >>
> > > > >> > Also, the latest linux kernel still has the X-Gene storm 
> > > > >> > distributor
> > > > >> > address as "0x78010000" in the device tree, which is what the Xen 
> > > > >> > code
> > > > >> > considers a match with the old firmware.  What were the addresses 
> > > > >> > for
> > > > >> > the device tree supposed to be changed to?
> > > > >>
> > > > >> We usually don't care, as the GIC address is provided by the
> > > > >> bootloader,
> > > > >> whether via DT or ACPI (this is certainly what happens on Mustang).
> > > > >> Whatever is still in the kernel tree is just as dead as the platform
> > > > >> it
> > > > >> describes.
> > > > >>
> > > > >> > Is my understanding
> > > > >> > correct that there is a different base address required to access 
> > > > >> > the
> > > > >> > "non-secure" region instead of the "secure" 0x78010000 region?  I'm
> > > > >> > trying to see if there are corresponding different addresses for 
> > > > >> > the
> > > > >> > keystone K2E, but haven't found them yet in the manuals.
> > > > >>
> > > > >> There is no such address. Think of the NS bit as an *address space*
> > > > >> identifier.
> > > > >>
> > > > >> The only reason XGene presents the NS part of the GIC at a different
> > > > >> address is because XGene is broken enough not to have EL3, hence no
> > > > >> secure mode. To wire the GIC (and other standard ARM IPs) to the 
> > > > >> core,
> > > > >> the designers simply used the CPU NS signal as an address bit.
> > > > >>
> > > > >> On your platform, the NS bit does exist. I strongly suppose that it
> > > > >> isn't wired to the GIC. Please talk to your SoC vendor for whether 
> > > > >> iot
> > > > >> is possible to work around this.
> > > > >>
> > > > > I do have a question about this out to TI, but at least this method
> > > > > gives me something to work with in the meantime.  I was just looking
> > > > > to confirm that there wouldn't be any other undesirable side effects
> > > > > with Dom0 or DomU when using it.  Was there an actual FPGA for the
> > > > > X-Gene that needed to be updated which controlled the GIC access?  Or
> > > > > by firmware do you mean the boot loader (e.g. uboot).  Thanks for the
> > > > > support so far to all.
> > > >
> > > > As I said, the specific case of XGene was just a matter of picking the
> > > > right address, as the NS bit is used as an address bit on this platform.
> > > > This was possible because this machine doesn't have any form of
> > > > security. So no HW was changed, no FPGA reprogrammed. Only a firmware
> > > > table was fixed to point to the right spot. Not even u-boot or EFI was
> > > > changed.
> > > Ok, thank you for clarifying.  I have one more question if you don't
> > > mind.  I'm aware that dom0 can share physical memory with dom1 via
> > > grant tables.
> > > However, is it possible to reserve a chunk of contiguous physical
> > > memory and directly allocate it only to dom1?
> > > For example, if I wanted dom1 to have access to 8MB of contiguous
> > > memory at 0x8200_0000 (in addition to whatever virtual memory Xen
> > > gives it).
> > > How would one go about doing this on ARM?  Is there something in the
> > > guest config or device tree that can be set?  Thanks for you help.
> >
> > There isn't a "proper" way to do it, only a workaround.
> >
> > It is possible to do it by using the iomem option, which is meant for
> > device MMIO regions.
> >
> > We have patches in the Xilinx Xen tree (not upstream) to allow for
> > specifying the cacheability that you want for the iomem mapping so that
> > you can map it as normal memory. This is the latest branch:
> >
> > https://github.com/Xilinx/xen.git xilinx/release-2020.1
> >
> > The relevant commits are the ones from:
> > https://github.com/Xilinx/xen/commit/a5c76ac1c5dc14d3e9ccedc5c1e7dd2ddc1397b6
> > to:
> > https://github.com/Xilinx/xen/commit/b4b7e91c1524f9cf530b81b7ba95df2bf50c78e4
> >
> > You might want to make sure that the page is not used by the normal
> > memory allocator. This document explains how to something along those
> > lines:
> >
> > https://github.com/Xilinx/xen/commit/35f72d9130448272e07466bd73cc30406f33135e
>
> Thank you.  I appreciate it.