Re: [Xen-devel] RFC: [PATCH 1/3] Enhance platform support for PCI

["Jan Beulich" <JBeulich@xxxxxxxx>]

>>> On 20.02.15 at 18:33, <ian.campbell@xxxxxxxxxx> wrote:
> On Fri, 2015-02-20 at 15:15 +0000, Jan Beulich wrote:
>> > That's the issue we are trying to resolve, with device tree there is no
>> > explicit segment ID, so we have an essentially unindexed set of PCI
>> > buses in both Xen and dom0.
>> 
>> How that? What if two bus numbers are equal? There ought to be
>> some kind of topology information. Or if all buses are distinct, then
>> you don't need a segment number.
> 
> It's very possible that I simply don't have the PCI terminology straight
> in my head, leading to me talking nonsense.
> 
> I'll explain how I'm using it and perhaps you can put me straight...
> 
> My understanding was that a PCI segment equates to a PCI host
> controller, i.e. a specific instance of some PCI host IP on an SoC.

No - there can be multiple roots (i.e. host bridges) on a single
segment. Segments are - afaict - purely a scalability extension
allowing to overcome the 256 bus limit.

> I
> think that PCI specs etc perhaps call a segment a PCI domain, which we
> avoided in Xen due to the obvious potential for confusion.

Right, the two terms are getting mixed depending on where you
look.

> A PCI host controller defines the root of a bus, within which the BDF
> need not be distinct due to the differing segments which are effectively
> a higher level namespace on the BDFs.

The host controller really defines the root of a tree (often covering
multiple buses, i.e. as soon as bridges come into play).

> So given a system with two PCI host controllers we end up with two
> segments (lets say A and B, but choosing those is the topic of this
> thread) and it is acceptable for both to contain a bus 0 with a device 1
> on it, i.e. (A:0:0.0) and (B:0:0.0) are distinct and can coexist.
> 
> It sounds like you are saying that this is not actually acceptable and
> that 0:0.0 must be unique in the system irrespective of the associated
> segment? iow (B:0:0.0) must be e.g. (B:1:0.0) instead?

No, there can be multiple buses numbered zero. And at the same
time a root bus doesn't need to be bus zero on its segment.

> Just for reference a DT node describing a PCI host controller might look
> like (taking the APM Mustang one as an example):
> 
>                 pcie0: pcie@1f2b0000 {
>                         status = "disabled";
>                         device_type = "pci";
>                         compatible = "apm,xgene-storm-pcie", "apm,xgene-pcie";
>                         #interrupt-cells = <1>;
>                         #size-cells = <2>;
>                         #address-cells = <3>;
>                         reg = < 0x00 0x1f2b0000 0x0 0x00010000   /* 
> Controller registers */
>                                 0xe0 0xd0000000 0x0 0x00040000>; /* PCI 
> config space */
>                         reg-names = "csr", "cfg";
>                         ranges = <0x01000000 0x00 0x00000000 0xe0 0x10000000 
> 0x00 0x00010000   /* io */
>                                   0x02000000 0x00 0x80000000 0xe1 0x80000000 
> 0x00 0x80000000>; /* mem */
>                         dma-ranges = <0x42000000 0x80 0x00000000 0x80 
> 0x00000000 0x00 0x80000000
>                                       0x42000000 0x00 0x00000000 0x00 
> 0x00000000 0x80 0x00000000>;
>                         interrupt-map-mask = <0x0 0x0 0x0 0x7>;
>                         interrupt-map = <0x0 0x0 0x0 0x1 &gic 0x0 0xc2 0x1
>                                          0x0 0x0 0x0 0x2 &gic 0x0 0xc3 0x1
>                                          0x0 0x0 0x0 0x3 &gic 0x0 0xc4 0x1
>                                          0x0 0x0 0x0 0x4 &gic 0x0 0xc5 0x1>;
>                         dma-coherent;
>                         clocks = <&pcie0clk 0>;
>                 };
> 
> I expect most of this is uninteresting but the key thing is that there
> is no segment number nor topology relative to e.g. "pcie1:
> pcie@1f2c0000" (the node look identical except e.g. all the base
> addresses and interrupt numbers differ).

What I don't get from this is where the BDF is being represented.
Yet that arrangement is fundamental to understand whether you
really need segments to properly disambiguate devices.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel