Re: [Xen-devel] [RFC PATCH 07/12] hvmloader: allocate MMCONFIG area in the MMIO hole + minor code refactoring

[Alexey G <x1917x@xxxxxxxxx>]

On Wed, 21 Mar 2018 17:15:04 +0000
Roger Pau Monné <roger.pau@xxxxxxxxxx> wrote:
[...]
>> Above scenario makes it obvious that at least for QEMU the MMIO->PCI
>> conf translation is a redundant step. Why not to allow specifying
>> for DM whether it prefers to receive MMCONFIG accesses as native
>> (MMIO ones) or as translated PCI conf ioreqs?  
>
>You are just adding an extra level of complexity to an interface
>that's fairly simple. You register a PCI device using
>XEN_DMOP_IO_RANGE_PCI and you get IOREQ_TYPE_PCI_CONFIG ioreqs.

Yes, and it is still needed as we have two distinct (and not equal)
interfaces to PCI conf space. Apart from 0..FFh range overlapping they
can be considered very different interfaces. And whether it is a real
system or emulated -- we can use either one of these two interfaces or
both.

For QEMU zero changes are needed to support MMCONFIG MMIO accesses if
they come as MMIO ioreqs. It's just what its MMCONFIG emulation code
expects.
Anyway, for (kind of vague) users of the multiple ioreq servers
capability we can enable MMIO translation to PCI conf ioreqs. Note that
actually this is an extra step, not forwarding trapped MMCONFIG MMIO
accesses to the selected device model as is.

>Getting both IOREQ_TYPE_PCI_CONFIG and IOREQ_TYPE_COPY for PCI config
>space access is misleading.

These are very different accesses, both in transport and capabilities.

>In both cases Xen would have to do the MCFG access decoding in order
>to figure out which IOREQ server will handle the request. At which
>point the only step that you avoid is the reconstruction of the memory
>access from the IOREQ_TYPE_PCI_CONFIG which is trivial.

The "reconstruction of the memory access" you mentioned won't be easy
actually. The thing is, address_space_read/write is not all what we
need.

In order to translate PCI conf ioreqs back to emulated MMIO ops, we
need to be an involved party, mainly to know where MMCONFIG area is
located so we can construct the address within its range from BDF.
This piece of information is destroyed in the process of MMIO ioreq
translation to PCI conf type.

The code which parse PCI conf ioreqs in xen-hvm.c doesn't know anything
about the current emulated MMCONFIG state. The correct way to have this
info is to participate in its emulation. As we don't participate, we
have no other way than trying to gain backdoor access to PCIHost fields
via things like object_resolve_*(). This solution is cumbersome and
ugly but will work... and may break anytime due to changes in QEMU. 

QEMU maintainers will grin while looking at all this I'm afraid --
trapped MMIO accesses which are translated to PCI conf accesses which
in turn translated back to emulated MMIO accesses upon receiving, along
with tedious attempts to gain access to MMCONFIG-related info as we're
not invited to the MMCONFIG emulation party.

The more I think about it, the more I like the existing
map_io_range_to_ioreq_server() approach. :( It works without doing
anything, no hacks, no new interfaces, both MMCONFIG and CF8/CFC are
working as expected. There is a problem to make it compatible with
the specific multiple ioreq servers feature, but providing a new
dmop/hypercall (which you suggest is a must have thing to trap MMCONFIG
MMIO to give QEMU only the freedom to tell where it is located) allows
to solve this problem in any possible way, either MMIO -> PCI conf
translation or anything else.

>> We can still route either ioreq
>> type to multiple device emulators accordingly.  
>
>It's exactly the same that's done for IO space PCI config space
>addresses. QEMU gets an IOREQ_TYPE_PCI_CONFIG and it replays the IO
>space access using do_outp and cpu_ioreq_pio.

...And it is completely limited to basic PCI conf space. I don't know
the context of this line in xen-hvm.c:

val = (1u << 31) | ((req->addr & 0x0f00) << 16) | ((sbdf & 0xffff) << 8)
       | (req->addr & 0xfc);

but seems like current QEMU versions do not expect anything similar to
AMD ECS-style accesses for 0CF8h. It is limited to basic PCI conf only.

>If you think using IOREQ_TYPE_COPY for MCFG accesses is such a benefit
>for QEMU, why not just translate the IOREQ_TYPE_PCI_CONFIG into
>IOREQ_TYPE_COPY in handle_ioreq and dispatch it using
>cpu_ioreq_move?

Answered above, we need to somehow have access to the info which don't
belong to us for this step.

>Thanks, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel