Re: [Xen-devel] PVH CPU hotplug design document

[Roger Pau Monné <roger.pau@xxxxxxxxxx>]

On Mon, Jan 16, 2017 at 09:50:53AM -0700, Jan Beulich wrote:
> >>> On 16.01.17 at 17:31, <roger.pau@xxxxxxxxxx> wrote:
> > On Mon, Jan 16, 2017 at 09:09:55AM -0700, Jan Beulich wrote:
> >> >>> On 16.01.17 at 16:14, <roger.pau@xxxxxxxxxx> wrote:
> >> > On Fri, Jan 13, 2017 at 08:51:57AM -0700, Jan Beulich wrote:
> >> >> >>> On 12.01.17 at 13:13, <roger.pau@xxxxxxxxxx> wrote:
> >> >> > # Introduction
> >> >> > 
> >> >> > One of the design goals of PVH is to be able to remove as much Xen PV 
> >> >> > specific
> >> >> > code as possible, thus limiting the number of Xen PV interfaces used 
> >> >> > by guests,
> >> >> > and tending to use native interfaces (as used by bare metal) as much 
> >> >> > as
> >> >> > possible. This is in line with the efforts also done by Xen on ARM 
> >> >> > and helps
> >> >> > reduce the burden of maintaining huge amounts of Xen PV code inside 
> >> >> > of guests
> >> >> > kernels.
> >> >> > 
> >> >> > This however presents some challenges due to the model used by the Xen
> >> >> > Hypervisor, where some devices are handled by Xen while others are 
> >> >> > left for the
> >> >> > hardware domain to manage. The fact that Xen lacks and AML parser 
> >> >> > also makes it
> >> >> > harder, since it cannot get the full hardware description from 
> >> >> > dynamic ACPI
> >> >> > tables (DSDT, SSDT) without the hardware domain collaboration.
> >> >> 
> >> >> Considering all the difficulties with the proposed model plus the little
> >> >> code the PV vCPU hotplug logic requires in the kernel (assuming a
> >> >> xenbus driver to be there anyway), I'm rather unconvinced that
> >> >> going this route instead of sticking to the PV model is actually
> >> >> desirable. And clearly, for consistency within the kernel, in such a
> >> >> case I'd then also favor sticking to this model for DomU.
> >> > 
> >> > We would at least have to pass the APIC ID in order to perform vCPU 
> >> > hotplug for
> >> > PVH, and the ACPI spec mandates that when using x2APIC structures in the 
> >> > MADT,
> >> > there must be a matching processor object in the DSDT (5.2.12.12).
> >> > 
> >> > Declaring processor objects in the DSDT won't be possible for Xen, but 
> >> > we can
> >> > at least declare them in a SSDT, which seems better than not doing it at 
> >> > all.
> >> > Maybe we can get ACPI to loosen the spec and don't mandate DSDT anymore.
> >> 
> >> I don't understand this reply of yours: How do any ACPI requirements
> >> come into play when using the PV hotplug mechanism for vCPU-s?
> > 
> > This clearly isn't a requirement when doing PV vCPU hotplug, but it's a
> > violation of the spec (proving x2APIC entries without matching processor
> > objects), so I wouldn't be surprised if ACPICA or any other ACPI 
> > implementation
> > refuses to boot on systems with x2APIC entries but no processor objects.
> 
> Good point, but what do you suggest short of declaring PVH v2 Dom0
> impossible to properly implement? I think that the idea of multiplexing
> ACPI for different purposes is simply going too far. For PV there's no
> such problem, as the Dom0 OS is expected to be aware that processor
> information coming from ACPI is not applicable to the view on CPUs it
> has (read: vCPU-s). And therefore, unless clean multiplexing is possible,
> I think PVH will need to retain this requirement (at which point there's
> no spec violation anymore).

But we definitely want to use ACPI to pass the boot vCPU information, using the
MADT for both DomU and Dom0.

Then for PVH DomU using ACPI vCPU hotplug makes perfect sense, it requires less
Xen specific code in the OS and it's fairly easy to implement inside of
Xen/toolstack. But I understand that using different methods for DomU vs Dom0
is very awkward. I still think that ACPI vCPU hotplug for Dom0 this is not so
far-fetched, and that it could be doable.

Could we introduce a new CPUID flag to notify the guest of whether it should
expect ACPI vCPU hotplug or PV vCPU hotplug?

I don't really like having Xen-specific checks inside of OSes, like "it's PVH
guest" then short circuiting a bunch of native logic. For example the ACPICA
ACPI shutdown hooks for Xen Dom0 never made it upstream, and it's very hard for
me to argue with the FreeBSD ACPICA maintainer about why those are needed,
and why he has to maintain a patch on top of upstream ACPICA only for Xen.

I will nonetheless send a new version of the design document, adding the
comments that I have received on the last draft.

Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel