[Xen-devel] [PATCH v3 5/6] xen/arm: update the docs about heterogeneous computing

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

Introduce a new document about big.LITTLE and update the documentation
of hmp-unsafe.

Also update the warning messages to point users to the docs.

Signed-off-by: Stefano Stabellini <sstabellini@xxxxxxxxxx>
CC: jbeulich@xxxxxxxx
CC: konrad.wilk@xxxxxxxxxx
CC: tim@xxxxxxx
CC: wei.liu2@xxxxxxxxxx
CC: andrew.cooper3@xxxxxxxxxx
CC: George.Dunlap@xxxxxxxxxxxxx
CC: ian.jackson@xxxxxxxxxxxxx

---

Changes in v3:
- split warning messages to be under 72 chars

Changes in v2:
- add a separate doc for big.LITTLE
- improve the warning message
---
 docs/misc/arm/big.LITTLE.txt        | 46 +++++++++++++++++++++++++++++++++++++
 docs/misc/xen-command-line.markdown |  7 +++++-
 xen/arch/arm/smpboot.c              | 11 +++++----
 3 files changed, 59 insertions(+), 5 deletions(-)
 create mode 100644 docs/misc/arm/big.LITTLE.txt

diff --git a/docs/misc/arm/big.LITTLE.txt b/docs/misc/arm/big.LITTLE.txt
new file mode 100644
index 0000000..b6ea1c9
--- /dev/null
+++ b/docs/misc/arm/big.LITTLE.txt
@@ -0,0 +1,46 @@
+big.LITTLE is a form of heterogeneous computing that comes with two
+types of general purpose cpu cores: big cores, more powerful and with a
+higher power consumption rate, and LITTLE cores, less powerful and
+cheaper to run. For example, Cortex A53 and Cortex A57 cpus. Typically,
+big cores are only recommended for burst activity, especially in
+battery powered environments. Please note that Xen doesn't not use any
+board specific power management techniques at the moment, it only uses
+WFI. It is recommended to check the vendor's big.LITTLE and power
+management documentation before using it in a Xen environment.
+
+
+big and LITTLE cores are fully compatible in terms of instruction sets,
+but can differ in many subtle ways. For example, their cacheline sizes
+might differ. For this reason, vcpu migration between big and LITTLE
+cores can lead to data corruptions.
+
+Today, the Xen scheduler does not have support for big.LITTLE,
+therefore, it might unknowingly move any vcpus between big and LITTLE
+cores, potentially leading to breakages. To avoid this kind of issues,
+at boot time Xen disables all cpus that differ from the boot cpu.
+
+
+Expert users can enable all big.LITTLE cores by passing hmp-unsafe=true
+to the Xen command line [1]. Given the lack of big.LITTLE support in the
+scheduler, it is only safe if the cpu affinity of all domains is
+manually specified, so that the scheduler is not allowed to switch a
+vcpu from big to LITTLE or vice versa.
+
+In the case of dom0, dom0_vcpus_pin needs to be added to the Xen command
+line options [1]. For DomUs, the `cpus' option should be added to all VM
+config files [2].
+
+For example, if the first 4 cpus are big and the last 4 are LITTLE, the
+following options run all domain vcpus on either big or LITTLE cores
+(not both):
+
+  cpus = "0-3"
+  cpus = "4-7"
+
+The following option runs one domain vcpu as big and one as LITTLE:
+
+  cpus = ["0-3", "4-7"]
+
+
+[1] docs/misc/xen-command-line.markdown
+[2] docs/man/xl.cfg.pod.5
diff --git a/docs/misc/xen-command-line.markdown 
b/docs/misc/xen-command-line.markdown
index 7b80119..4391b75 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -992,7 +992,12 @@ Control Xens use of the APEI Hardware Error Source Table, 
should one be found.
 
 Say yes at your own risk if you want to enable heterogenous computing
 (such as big.LITTLE). This may result to an unstable and insecure
-platform. When the option is disabled (default), CPUs that are not
+platform, unless you manually specify the cpu affinity of all domains so
+that all vcpus are scheduled on the same class of pcpus (big or LITTLE
+but not both). vcpu migration between big cores and LITTLE cores is not
+supported. See docs/misc/arm/big.LITTLE.txt for more information.
+
+When the hmp-unsafe option is disabled (default), CPUs that are not
 identical to the boot CPU will be parked and not used by Xen.
 
 ### hpetbroadcast
diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
index 122c0b5..04efb33 100644
--- a/xen/arch/arm/smpboot.c
+++ b/xen/arch/arm/smpboot.c
@@ -266,7 +266,8 @@ void __init smp_init_cpus(void)
 
     if ( opt_hmp_unsafe )
         warning_add("WARNING: HMP COMPUTING HAS BEEN ENABLED.\n"
-                    "It has implications on the security and stability of the 
system.\n");
+                    "It has implications on the security and stability of the 
system,\n"
+                    "unless the cpu affinity of all domains is specified.\n");
 }
 
 int __init
@@ -308,13 +309,15 @@ void start_secondary(unsigned long boot_phys_offset,
     /*
      * Currently Xen assumes the platform has only one kind of CPUs.
      * This assumption does not hold on big.LITTLE platform and may
-     * result to instability and insecure platform. Better to park them
-     * for now.
+     * result to instability and insecure platform (unless cpu affinity
+     * is manually specified for all domains). Better to park them for
+     * now.
      */
     if ( !opt_hmp_unsafe &&
          current_cpu_data.midr.bits != boot_cpu_data.midr.bits )
     {
-        printk(XENLOG_ERR "CPU%u MIDR (0x%x) does not match boot CPU MIDR 
(0x%x).\n",
+        printk(XENLOG_ERR "CPU%u MIDR (0x%x) does not match boot CPU MIDR 
(0x%x),\n"
+               "disable cpu (see big.LITTLE.txt under docs/).\n",
                smp_processor_id(), current_cpu_data.midr.bits,
                boot_cpu_data.midr.bits);
         stop_cpu();
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel