[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] [early RFC] ARM PCI Passthrough design document

On Wed, Jan 25, 2017 at 06:53:20PM +0000, Julien Grall wrote:
> Hi Stefano,
> On 24/01/17 20:07, Stefano Stabellini wrote:
> > On Tue, 24 Jan 2017, Julien Grall wrote:
> > > > > whilst for Device Tree the segment number is not available.
> > > > > 
> > > > > So Xen needs to rely on DOM0 to discover the host bridges and notify 
> > > > > Xen
> > > > > with all the relevant informations. This will be done via a new 
> > > > > hypercall
> > > > > PHYSDEVOP_pci_host_bridge_add. The layout of the structure will be:
> > > > 
> > > > I understand that the main purpose of this hypercall is to get Xen and 
> > > > Dom0
> > > > to
> > > > agree on the segment numbers, but why is it necessary? If Dom0 has an
> > > > emulated contoller like any other guest, do we care what segment numbers
> > > > Dom0 will use?
> > > 
> > > I was not planning to have a emulated controller for DOM0. The physical 
> > > one is
> > > not necessarily ECAM compliant so we would have to either emulate the 
> > > physical
> > > one (meaning multiple different emulation) or an ECAM compliant.
> > > 
> > > The latter is not possible because you don't know if there is enough free 
> > > MMIO
> > > space for the emulation.
> > > 
> > > In the case on ARM, I don't see much the point to emulate the host bridge 
> > > for
> > > DOM0. The only thing we need in Xen is to access the configuration space, 
> > > we
> > > don't have about driving the host bridge. So I would let DOM0 dealing with
> > > that.
> > > 
> > > Also, I don't see any reason for ARM to trap DOM0 configuration space 
> > > access.
> > > The MSI will be configured using the interrupt controller and it is a 
> > > trusted
> > > Domain.
> > 
> > These last you sentences raise a lot of questions. Maybe I am missing
> > something. You might want to clarify the strategy for Dom0 and DomUs,
> > and how they differ, in the next version of the doc.
> > 
> > At some point you wrote "Instantiation of a specific driver for the host
> > controller can be easily done if Xen has the information to detect it.
> > However, those drivers may require resources described in ASL." Does it
> > mean you plan to drive the physical host bridge from Xen and Dom0
> > simultaneously?
> I may miss some bits, so feel free to correct me if I am wrong.
> My understanding is host bridge can be divided in 2 parts:
>       - Initialization of the host bridge
>       - Access the configuration space
> For generic host bridge, the initialization is inexistent. However some host
> bridge (e.g xgene, xilinx) may require some specific setup and also
> configuring clocks. Given that Xen only requires to access the configuration
> space, I was thinking to let DOM0 initialization the host bridge. This would
> avoid to import a lot of code in Xen, however this means that we need to
> know when the host bridge has been initialized before accessing the
> configuration space.

Can the bridge be initialized without Dom0 having access to the ECAM area? If
that's possible I would do:

1. Dom0 initializes the bridge (whatever that involves).
2. Dom0 calls PHYSDEVOP_pci_mmcfg_reserved to register the bridge with Xen:
 2.1 Xen scans the bridge and detects the devices.
 2.2 Xen maps the ECAM area into Dom0 stage-2 p2m.
3. Dom0 scans the bridge &c (whatever is done on native).

> Now regarding the configuration space, I think we can divide in 2 category:
>       - indirect access, the configuration space are multiplexed. An example
> would be the legacy method on x86 (e.g 0xcf8 and 0xcfc). A similar method is
> used for x-gene PCI driver ([1]).
>       - ECAM like access, where each PCI configuration space will have it is 
> own
> address space. I said "ECAM like" because some host bridge will require some
> bits fiddling when accessing register (see thunder-ecam [2])
> There are also host bridges that mix both indirect access and ECAM like
> access depending on the device configuration space accessed (see thunder-pem
> [3]).

Hay! Sounds like fun...

> When using ECAM like host bridge, I don't think it will be an issue to have
> both DOM0 and Xen accessing configuration space at the same time. Although,
> we need to define who is doing what. In general case, DOM0 should not
> touched an assigned PCI device. The only possible interaction would be
> resetting a device (see my answer below).

Iff Xen is really going to perform the reset of passthrough devices, then I
don't see any reason to expose those devices to Dom0 at all, IMHO you should
hide them from ACPI and ideally prevent Dom0 from interacting with them using
the PCI configuration space (although that would require trapping on accesses
to the PCI config space, which AFAIK you would like to avoid).

> When using indirect access, we cannot let DOM0 and Xen accessing any PCI
> configuration space at the same time. So I think we would have to emulate
> the physical host controller.
> Unless we have a big requirement to trap DOM0 access to the configuration
> space, I would only keep the emulation to the strict minimum (e.g for
> indirect access) to avoid ending-up handling all the quirks for ECAM like
> host bridge.
> If we need to trap the configuration space, I would suggest the following
> for ECAM like host bridge:
>       - For physical host bridge that does not require initialization and is
> nearly ECAM compatible (e.g require register fiddling) => replace by a
> generic host bridge emulation for DOM0
>       - For physical host bridge that require initialization but is ECAM
> compatible (e.g AFAICT xilinx [4]) => trap the ECAM access but let DOM0
> handling the host bridge initialization
>       - For all other host bridges => I don't know if there are host bridges
> falling under this category. I also don't have any idea how to handle this.

Without knowing much about this it's hard for me to have an opinion. IMHO you
should prevent Dom0 from accessing the configuration space of devices it's not
supposed to manage (eg: passthrough), but since Dom0 is trusted I guess you can
find some other way to tell Dom0 to avoid poking at those devices, and avoid
trapping accesses to the configuration space.


Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.