Re: PCI passthrough: possible bug in memory relocation

[Jason Andryuk <jandryuk@xxxxxxxxx>]

On Mon, Apr 4, 2022 at 7:37 PM Mateusz <mati7337@xxxxxxxxxxxxx> wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
>
> > I'm afraid answering this requires debugging the issue. Yet you don't
> > share any technical details (as to how things don't work, logs, and
> > alike), and the provided link also doesn't look to point to any such
> > information (and as an aside I consider it somewhat unfriendly to
> > point at such a bug as an information source, not just for reference).
> > I'm pretty sure this code in hvmloader did work at some point, but
> > since it may be used quite rarely I could see that it might have got
> > broken.
>
> Thanks for responding!
> I only wanted to ask to see if maybe it's a known issue, but I guess not.
> I'll try to debug and fix it myself so that's why I haven't posted more
> technical details yet.

OpenXT manually configures the xl.cfg mmio_hole setting by reading the
PCI BAR sizes.  The (haskell) code is here:
https://github.com/OpenXT/manager/commit/33fef12b242e3cc9b46a32d07c84bc593ee537c9
.  It runs in dom0 and can access the PCI sysfs entries.

Trying to do this in QEMU in the stubdom is tricky.  Vanilla Xen
hotplugs the PV PCI devices to the stubdom and then hotplugs those
into QEMU with QMP devce_add.  (Qubes changed (or has a change
pending) to cold plugging the PV PCI devices to the stubdom, but they
are still hotplugged into QEMU via QMP device_add calls.)  You won't
know which PCI devices are applicable, but I guess in the stubdom you
can just assume all of the PV ones should be passed through.

It would be better for QEMU to do this itself, but hotplugging PCI
devices means it doesn't have the needed information during startup.

Looking at https://github.com/QubesOS/qubes-vmm-xen-stubdom-linux/pull/44
You look at /proc/iomem - I guess that works with the cold plugging of
PV PCI devices into the stubdom.  You only grab the first PCI device
and return after that - not the sum of all PCI devices detected.

libxl knows the devices that will be assigned at domU boot time.  So
it could do the calculation and adjust the mmio_hole size itself.
That doesn't help hotplugging, but it will be better than the
situation today.  libxl already does some PCI sysfs stuff, so it might
be a good place to solve this.

Regards,
Jason