Re: [PATCH 1/2] docs/designs/launch: hyperlaunch design document

[Roger Pau Monné <roger.pau@xxxxxxxxxx>]

On Thu, Mar 25, 2021 at 10:14:31AM +0100, George Dunlap wrote:
> 
> 
> > On Mar 25, 2021, at 8:32 AM, Roger Pau Monne <roger.pau@xxxxxxxxxx> wrote:
> > 
> > On Wed, Mar 24, 2021 at 05:53:26AM -0700, Christopher Clark wrote:
> >> On Wed, Mar 24, 2021 at 1:01 AM Roger Pau Monné <roger.pau@xxxxxxxxxx> 
> >> wrote:
> >>> 
> >>> On Tue, Mar 23, 2021 at 10:39:53AM -0700, Christopher Clark wrote:
> >>>> On Thu, Mar 18, 2021 at 9:43 AM Roger Pau Monné <roger.pau@xxxxxxxxxx> 
> >>>> wrote:
> >>> If you offload domain creation of guests with
> >>> pci-passthrough devices to a control domain and/or hardware domain,
> >>> I'm not sure I see the difference from normal domain creation, ie:
> >>> it's no longer something specific to hyperlaunch, as I could achieve
> >>> the same by using the existing xendomains init script.
> >> 
> >> So that's not what we've proposed, and hopefully not what we'll need to do.
> >> 
> >> Do you know if there is a need to perform work to support the
> >> assignment of PCI devices at the point of domain creation (ie. in
> >> domain_create), rather than handling it in a later step of domain
> >> configuration, prior to the domain being started?
> > 
> > So while I think you could indeed create a domain from the hypervisor
> > in a paused state and attach the pci devices later from a
> > control/hardware domain, I don't see much benefit in doing it. If you
> > need to end up waiting for a control/hardware domain to attach the
> > devices and unpause you might as well do the whole domain creation
> > from such control/hardware domain.
> 
> My understanding was that one of the primary advantages of domB was
> that you could compile and run arbitrary code in whatever language
> you wanted to, using already known tools.  If *all* we want to do is
> to assign some pre-defined specific BDFs to specific domains, then
> sure, we could add that capability to Xen.

Well, it's not so easy because we require QEMU or pciback ATM on x86
in order to do pci passthrough to guests, so assigning BDFs to
specific domains from the hypervisor would need to be done using vPCI
(which is not yet ready for unprivileged guest usage) and only support
HVM kind of guests strictly.

Doing passthrough to PV guests directly from the hypervisor is
impossible AFAICT without putting pciback inside of the hypervisor.

> But suppose you wanted
> to have the flexibility to handle changes in hardware at boot time?
> “Scan through the PCI bus and assign anything that looks like a
> network card to domNet, and anything that looks like a USB
> controller to domUSB” is something you could easily do in domB, but
> would be way too complicated to add to Xen.

Right, but then you might as well create the domain from domB instead
of doing it in the hypervisor?

I'm not arguing about not using domB, I just don't see the benefit of
creating a paused domain from the hypervisor that then requires the
intervention of a further domain (domB) in order to finish creation.
Won't it be simpler to just create the domain and attach the pci
devices from domB?

Thanks, Roger.