Re: [Xen-devel] PVH CPU hotplug design document

["Jan Beulich" <JBeulich@xxxxxxxx>]

>>> On 16.01.17 at 18:44, <roger.pau@xxxxxxxxxx> wrote:
> On Mon, Jan 16, 2017 at 09:50:53AM -0700, Jan Beulich wrote:
>> >>> On 16.01.17 at 17:31, <roger.pau@xxxxxxxxxx> wrote:
>> > On Mon, Jan 16, 2017 at 09:09:55AM -0700, Jan Beulich wrote:
>> >> >>> On 16.01.17 at 16:14, <roger.pau@xxxxxxxxxx> wrote:
>> >> > On Fri, Jan 13, 2017 at 08:51:57AM -0700, Jan Beulich wrote:
>> >> >> >>> On 12.01.17 at 13:13, <roger.pau@xxxxxxxxxx> wrote:
>> >> >> > # Introduction
>> >> >> > 
>> >> >> > One of the design goals of PVH is to be able to remove as much Xen 
>> >> >> > PV 
> specific
>> >> >> > code as possible, thus limiting the number of Xen PV interfaces used 
>> >> >> > by 
> guests,
>> >> >> > and tending to use native interfaces (as used by bare metal) as much 
>> >> >> > as
>> >> >> > possible. This is in line with the efforts also done by Xen on ARM 
>> >> >> > and 
> helps
>> >> >> > reduce the burden of maintaining huge amounts of Xen PV code inside 
>> >> >> > of 
> guests
>> >> >> > kernels.
>> >> >> > 
>> >> >> > This however presents some challenges due to the model used by the 
>> >> >> > Xen
>> >> >> > Hypervisor, where some devices are handled by Xen while others are 
>> >> >> > left 
> for the
>> >> >> > hardware domain to manage. The fact that Xen lacks and AML parser 
>> >> >> > also 
> makes it
>> >> >> > harder, since it cannot get the full hardware description from 
>> >> >> > dynamic 
> ACPI
>> >> >> > tables (DSDT, SSDT) without the hardware domain collaboration.
>> >> >> 
>> >> >> Considering all the difficulties with the proposed model plus the 
>> >> >> little
>> >> >> code the PV vCPU hotplug logic requires in the kernel (assuming a
>> >> >> xenbus driver to be there anyway), I'm rather unconvinced that
>> >> >> going this route instead of sticking to the PV model is actually
>> >> >> desirable. And clearly, for consistency within the kernel, in such a
>> >> >> case I'd then also favor sticking to this model for DomU.
>> >> > 
>> >> > We would at least have to pass the APIC ID in order to perform vCPU 
> hotplug for
>> >> > PVH, and the ACPI spec mandates that when using x2APIC structures in 
>> >> > the 
> MADT,
>> >> > there must be a matching processor object in the DSDT (5.2.12.12).
>> >> > 
>> >> > Declaring processor objects in the DSDT won't be possible for Xen, but 
>> >> > we 
> can
>> >> > at least declare them in a SSDT, which seems better than not doing it 
>> >> > at 
> all.
>> >> > Maybe we can get ACPI to loosen the spec and don't mandate DSDT anymore.
>> >> 
>> >> I don't understand this reply of yours: How do any ACPI requirements
>> >> come into play when using the PV hotplug mechanism for vCPU-s?
>> > 
>> > This clearly isn't a requirement when doing PV vCPU hotplug, but it's a
>> > violation of the spec (proving x2APIC entries without matching processor
>> > objects), so I wouldn't be surprised if ACPICA or any other ACPI 
> implementation
>> > refuses to boot on systems with x2APIC entries but no processor objects.
>> 
>> Good point, but what do you suggest short of declaring PVH v2 Dom0
>> impossible to properly implement? I think that the idea of multiplexing
>> ACPI for different purposes is simply going too far. For PV there's no
>> such problem, as the Dom0 OS is expected to be aware that processor
>> information coming from ACPI is not applicable to the view on CPUs it
>> has (read: vCPU-s). And therefore, unless clean multiplexing is possible,
>> I think PVH will need to retain this requirement (at which point there's
>> no spec violation anymore).
> 
> But we definitely want to use ACPI to pass the boot vCPU information, using 
> the
> MADT for both DomU and Dom0.

Is that really set in stone?

> Then for PVH DomU using ACPI vCPU hotplug makes perfect sense, it requires 
> less
> Xen specific code in the OS and it's fairly easy to implement inside of
> Xen/toolstack. But I understand that using different methods for DomU vs Dom0
> is very awkward. I still think that ACPI vCPU hotplug for Dom0 this is not so
> far-fetched, and that it could be doable.
> 
> Could we introduce a new CPUID flag to notify the guest of whether it should
> expect ACPI vCPU hotplug or PV vCPU hotplug?

That would be an easy addition.

> I don't really like having Xen-specific checks inside of OSes, like "it's PVH
> guest" then short circuiting a bunch of native logic. For example the ACPICA
> ACPI shutdown hooks for Xen Dom0 never made it upstream, and it's very hard 
> for
> me to argue with the FreeBSD ACPICA maintainer about why those are needed,
> and why he has to maintain a patch on top of upstream ACPICA only for Xen.

I understand all those concerns, but we shouldn't replace one ugliness
by another. I.e. without a reasonably clean concept of how to use
ACPI here I can't help thinking that the PV model here is the cleaner one
despite the (little) extra code it requires in OSes.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel