Re: [PATCH RFC v2] vPCI: account for hidden devices

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Thu, 25 May 2023, Jan Beulich wrote:
> On 25.05.2023 01:37, Stefano Stabellini wrote:
> > On Wed, 24 May 2023, Jan Beulich wrote:
> >>>> RFC: _setup_hwdom_pci_devices()' loop may want splitting: For
> >>>>      modify_bars() to consistently respect BARs of hidden devices while
> >>>>      setting up "normal" ones (i.e. to avoid as much as possible the
> >>>>      "continue" path introduced here), setting up of the former may want
> >>>>      doing first.
> >>>
> >>> But BARs of hidden devices should be mapped into dom0 physmap?
> >>
> >> Yes.
> > 
> > The BARs would be mapped read-only (not read-write), right? Otherwise we
> > let dom0 access devices that belong to Xen, which doesn't seem like a
> > good idea.
> > 
> > But even if we map the BARs read-only, what is the benefit of mapping
> > them to Dom0? If Dom0 loads a driver for it and the driver wants to
> > initialize the device, the driver will crash because the MMIO region is
> > read-only instead of read-write, right?
> > 
> > How does this device hiding work for dom0? How does dom0 know not to
> > access a device that is present on the PCI bus but is used by Xen?
> 
> None of these are new questions - this has all been this way for PV Dom0,
> and so far we've limped along quite okay. That's not to say that we
> shouldn't improve things if we can, but that first requires ideas as to
> how.

For PV, that was OK because PV requires extensive guest modifications
anyway. We only run Linux and few BSDs as Dom0. So, making the interface
cleaner and reducing guest changes is nice-to-have but not critical.

For PVH, this is different. One of the top reasons for AMD to work on
PVH is to enable arbitrary non-Linux OSes as Dom0 (when paired with
dom0less/hyperlaunch). It could be anything from Zephyr to a
proprietary RTOS like VxWorks. Minimal guest changes for advanced
features (e.g. Dom0 S3) might be OK but in general I think we should aim
at (almost) zero guest changes. On ARM, it is already the case (with some
non-upstream patches for dom0less PCI.)

For this specific patch, which is necessary to enable PVH on AMD x86 in
gitlab-ci, we can do anything we want to make it move faster. But
medium/long term I think we should try to make non-Xen-aware PVH Dom0
possible.

Cheers,

Stefano