Re: [Xen-devel] [RFC PATCH 07/12] hvmloader: allocate MMCONFIG area in the MMIO hole + minor code refactoring

[Roger Pau Monné <roger.pau@xxxxxxxxxx>]

On Wed, Mar 28, 2018 at 01:37:29AM +1000, Alexey G wrote:
> On Tue, 27 Mar 2018 09:45:30 +0100
> Roger Pau Monné <roger.pau@xxxxxxxxxx> wrote:
> 
> >On Tue, Mar 27, 2018 at 05:42:11AM +1000, Alexey G wrote:
> >> On Mon, 26 Mar 2018 10:24:38 +0100
> >> Roger Pau Monné <roger.pau@xxxxxxxxxx> wrote:
> >>   
> >> >On Sat, Mar 24, 2018 at 08:32:44AM +1000, Alexey G wrote:  

> BTW, another somewhat related problem at the moment is that Xen knows
> nothing about a chipset-specific MMIO hole(s). Due to this, it is
> possible for a guest to map PT BARs outside the MMIO hole, leading to
> errors like this:
> 
> (XEN) memory_map:remove: dom4 gfn=c8000 mfn=c8000 nr=2000
> (XEN) memory_map:add: dom4 gfn=ffffffffc8000 mfn=c8000 nr=2000
> (XEN) p2m.c:1121:d0v5 p2m_set_entry: 0xffffffffc8000:9 -> -22 (0xc8000)
> (XEN) memory_map:fail: dom4 gfn=ffffffffc8000 mfn=c8000 nr=2000 ret:-22
> (XEN) memory_map:remove: dom4 gfn=ffffffffc8000 mfn=c8000 nr=2000
> (XEN) p2m.c:1228:d0v5 gfn_to_mfn failed! gfn=ffffffffc8000 type:4
> (XEN) memory_map: error -22 removing dom4 access to [c8000,c9fff]
> (XEN) memory_map:remove: dom4 gfn=ffffffffc8000 mfn=c8000 nr=2000
> (XEN) p2m.c:1228:d0v5 gfn_to_mfn failed! gfn=ffffffffc8000 type:4
> (XEN) memory_map: error -22 removing dom4 access to [c8000,c9fff]
> (XEN) memory_map:add: dom4 gfn=c8000 mfn=c8000 nr=2000
> 
> Note that it was merely a lame BAR sizing attempt from the guest-side SW
> (a PCI config space viewing tool) -- writing F's to the high part of the
> MMIO BAR first.

You should disable memory decoding before attempting to size a BAR.

This error has nothing to do with trying to move a BAR outside of the
MMIO hole, this error is caused by the gfn being bigger than the guest
physical address width AFAICT.

> If we will know the guest's MMIO hole bounds, we can adapt to this
> behavior, avoiding erroneous mapping attempts to a wrong address
> outside the MMIO hole. Only the MMIO hole designated range can be used
> to map PT device BARs.
> 
> So, if we will be actually emulating MCH's MMIO hole related registers
> in Xen as well -- we can use them as scratchpad registers (write-once
> of course) to pass this kind of information between Xen and other
> involved parties as an alternative to eg. a dedicated hypercall.

I'm not sure where this information is stored in MCH, guest OSes tend
to fetch this from the ACPI _CRS method of the host-pci bridge device.

I also don't see QEMU emulating such registers, but yes, I won't be
opposed to storing/reporting this in some registers if that's indeed
supported. Note that I don't think this should be mandatory for adding
Q35 support though.

> >> What this approach will require:
> >> --------------------------------
> >> 
> >> - Changes in QEMU code to support a new chipset-less machine(s). In
> >>   theory might be possible to implement on top of the "null" machine
> >>   concept  
> >
> >Not all parts of the chipset should go inside of Xen, ATM I only
> >foresee Q35 MCH being implemented inside of Xen. So I'm not sure
> >calling this a chipset-less machine is correct from QEMU PoV.
> 
> Emulating only MCH in Xen will still require lot of changes but 
> overall benefit will become unclear -- basically, we just move
> PCIEXBAR emulation to Xen from QEMU.

At least it would make Xen the one controlling the MCFG area, which is
important. It would also be the first step into moving other chipset
functionality into Xen.

Not doing it just perpetuates the bad precedent that we already have
with the previous chipset.

> >What are specifically the registers that you mention?
> 
> Write-once emulation of TOLUD/TOUUD/REMAPBASE/REMAPLIMIT registers for
> hvmloader to use. That's the approach I'm actually using to make
> 'hvmloader/allow-memory-relocate=1' to work. Memory relocation without
> relying on add_to_physmap hypercall for hvmloader (which it does
> currently) while having MMIO/memory layout synchronized between all
> parties. There are multiple benefits (mostly for PT needs), including
> the MMIO hole auto-sizing support but this approach won't be accepted
> well with "toolstack should do everything" attitude I'm afraid.

You seem to be trying to fix several issues at the same time, which
just makes this much more complex than needed. The initial aim of this
series was to allow HVM guests to use the Q35 chipset. I think that's
what we should focus on.

As you have listed above (and in other emails) there are many
limitations with the current HVM approach, which I would be more than
happy for you to solve. But IMO not all of them must be solved in
order to add Q35 support.

Since this series and email thread has already gone quite far, would
you mind writing a design document with the approach that we
discussed?

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel