[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [PATCH v1 2/6] xen/arm: Reserve resources for virtio-pci
Hi Stefano, On 25/09/2024 00:16, Stefano Stabellini wrote: On Tue, 24 Sep 2024, Julien Grall wrote:On 24/09/2024 18:11, Edgar E. Iglesias wrote:On Tue, Sep 24, 2024 at 05:35:20PM +0100, Julien Grall wrote:Hi Edgar, On 24/09/2024 17:23, Edgar E. Iglesias wrote:From: "Edgar E. Iglesias" <edgar.iglesias@xxxxxxx> Reserve memory ranges and interrupt lines for an externally emulated PCI controller (e.g by QEMU) dedicated to hosting Virtio devices and potentially other emulated devices. Signed-off-by: Edgar E. Iglesias <edgar.iglesias@xxxxxxx> --- xen/include/public/arch-arm.h | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h index e19f0251a6..654b827715 100644 --- a/xen/include/public/arch-arm.h +++ b/xen/include/public/arch-arm.h @@ -494,6 +494,20 @@ typedef uint64_t xen_callback_t; #define GUEST_RAM1_BASE xen_mk_ullong(0x0200000000) /* 952GB of RAM @ 8GB */ #define GUEST_RAM1_SIZE xen_mk_ullong(0xee00000000) +/* Virtio PCI - Ordered by decreasing size to keep things aligned */ +#define GUEST_VIRTIO_PCI_PREFETCH_MEM_TYPE xen_mk_ullong(0x43000000) +#define GUEST_VIRTIO_PCI_PREFETCH_MEM_BASE xen_mk_ullong(0x0f000000000) +#define GUEST_VIRTIO_PCI_PREFETCH_MEM_SIZE xen_mk_ullong(0x100000000) + +#define GUEST_VIRTIO_PCI_ECAM_BASE (GUEST_VIRTIO_PCI_PREFETCH_MEM_BASE + \ + GUEST_VIRTIO_PCI_PREFETCH_MEM_SIZE) +#define GUEST_VIRTIO_PCI_ECAM_SIZE xen_mk_ullong(0x10000000) + +#define GUEST_VIRTIO_PCI_MEM_TYPE xen_mk_ullong(0x02000000) +#define GUEST_VIRTIO_PCI_MEM_BASE (GUEST_VIRTIO_PCI_ECAM_BASE + \ + GUEST_VIRTIO_PCI_ECAM_SIZE) +#define GUEST_VIRTIO_PCI_MEM_SIZE xen_mk_ullong(0x00002000000)Why is this specific to virtio PCI? However, I am not entirely convinced we should have a second PCI hostbridge exposed to the guest for a few reasons: 1. This require to reserve yet another range in the address space (could be solved with a more dynamic layout) 2. From your instructions, the guest needs to explicitly do a PCI rescan.Another big advantage I forgot to mention is disaggregation. In a world where the hostbridge is handled in Xen, you could have a PCI device emulated by dom0 while the other is emulated by a stub domain.So rather than having a second hostbridge, have you considered to extend the existing hostbridge (implemented in Xen) to support a mix of physical PCI device and virtual one?Thanks Julien, It's briefly come up in a couple of discussions but I haven't looked carefully at it. It is a good idea and it's probably worth prototyping to see what the gaps are in hypercall interfaces, QEMU support etc.I also vaguely recall to discuss it on xen-devel. But I couldn't find the discussion... :(. I think all the hypercalls should be there but will require some plumbing in Xen on Arm. QEMU should be able to request Xen to forward configuration access for a given PCI device (see XEN_DMOP_IO_RANGE_PCI). They will then be forwarded to QEMU using IOREQ_TYPE_PCI_CONFIG. We also have an hypercall to be able to inject interrupts from QEMU (see XEN_DMOP_set_irq_level).Hi Julien, Yes, I remember a thread on xen-devel too about this topic when EPAM suggested a similar two-hostbridges approach. I was one of the people suggesting to use a single hostbridge at the time. However, when we looked at the implementation more closely, the two-hostbridge approach was easier to get up and running. It works (almost) out of the box. Currently, we have the two-hostbridge solution working on both ARM and x86 to enable virtio-pci to work alongside vPCI in Dom0less/Hyperlaunch configurations. I understand this is the easiest solution... However, this requires code in Xen that I am not yet convinced it is good to have. I am not too concerned about the MMIO range part. This can be (easily) solved. I am more concerned about the support of background region and the fact the OS needs to be able to rescan. I am definitely not an expert of PCI, but AFAIK, it is possible to have the guest to be notified when a PCI device is hotplug. Why can't we use it? To me this raises the question why we should have a temporary solution upstream then?While I think that a single hostbridge is better architecturally, it is important to consider that virtio is moving toward a new transport (virtio-msg, Bertrand is also involved) which does not require a hostbridge. This new transport is key for all our use-cases as it addresses safety requirements and supports AMP configurations without a shared hypervisor between the frontend and backend. Edgar is one of the top contributors to virtio-msg. Given this, I don't think it's worthwhile to invest much effort in virtio-pci, as it will be replaced soon in embedded applications. Cheers, -- Julien Grall
|
Lists.xenproject.org is hosted with RackSpace, monitoring our |