[XenPPC] [PATCH] Linker script causes SMP memory corruption

[Amos Waterland <apw@xxxxxxxxxx>]

To understand the context for this patch, observe that the following is
the case in stock Xen/PPC:

 $ nm xen-syms | egrep ' per_cpu__|__per_cpu' | sort
 000000000044c680 d per_cpu__domain_shuttingdown
 000000000044c688 D per_cpu__mc_state
 000000000044c700 d per_cpu__t_timer
 ...
 000000000045cc80 D per_cpu__tlbflush_time
 000000000045d2b0 A __per_cpu_data_end
 000000000045d2b0 A __per_cpu_start
 0000000000460000 A __per_cpu_end

So __per_cpu_start and __per_cpu_data_end are identical, and none of the
per_cpu variable are actually between __per_cpu_start and __per_cpu_end.

When CPU #20 on a Xen/PPC SMP system tries to load or store to 
its t_timer per-cpu variable, a calculation like this will take place:

 address = 0x44c700 + (20 << 12)

So the address is 0x460700, which beyond __per_cpu_end among other
problems.  For my configuration, it lies in the __hypercall_table, so
when init_timer(per_cpu(t_timer)) is run, a portion of the hypercall
table is memset to zero and then some timer-related function pointers
are written to it.  Many cycles later, when init_hcalls is run, a
garbage pointer is dereferenced and a 0x200 exception occurs.

Also, the linker script has a bogus assumption that the per-cpu
section size can be calculated by adding (NR_CPUS << PERCPU_SHIFT) to
__per_cpu_start.  It needs to use __per_cpu_data_end instead, or a
different instance of memory corruption can occur.

Also, a .data.percpu section was not actually being created in the final
linked object.

This patch fixes the three issues described above.  Tested on
systemsim-gpul, JS20 and JS21.

Signed-off-by: Amos Waterland <apw@xxxxxxxxxx>

---

 arch/powerpc/xen.lds.S       |   15 +++++++++------
 include/asm-powerpc/percpu.h |    5 +++--
 2 files changed, 12 insertions(+), 8 deletions(-)

diff -r 539a1e666982 xen/arch/powerpc/xen.lds.S
--- a/xen/arch/powerpc/xen.lds.S        Fri Aug 18 14:07:50 2006 -0400
+++ b/xen/arch/powerpc/xen.lds.S        Sat Aug 19 01:53:55 2006 -0400
@@ -124,12 +124,15 @@ SECTIONS
   .inithcall.text : { *(.inithcall.text) }
   __inithcall_end = .;
 
-  __per_cpu_start = .;
-  .data.percpu : { *(.data.percpu) }
-  __per_cpu_data_end = .;
-  . = __per_cpu_start + (NR_CPUS << PERCPU_SHIFT);
-  . = ALIGN(STACK_SIZE);
-  __per_cpu_end = .;
+  .data.percpu : {
+    __per_cpu_start = .;
+    *(__per_cpu)
+    __per_cpu_data_end = .;
+    . = __per_cpu_data_end + (NR_CPUS << PERCPU_SHIFT);
+    __per_cpu_multiple_end = .; 
+    . = ALIGN(STACK_SIZE);
+    __per_cpu_end = .; 
+  }
 
   /* end Xen addition */
 
diff -r 539a1e666982 xen/include/asm-powerpc/percpu.h
--- a/xen/include/asm-powerpc/percpu.h  Fri Aug 18 14:07:50 2006 -0400
+++ b/xen/include/asm-powerpc/percpu.h  Sat Aug 19 01:07:21 2006 -0400
@@ -8,8 +8,9 @@
 
 /* Separate out the type, so (int[3], foo) works. */
 #define DEFINE_PER_CPU(type, name)                      \
-    __attribute__((__section__(".data.percpu")))        \
-    __typeof__(type) per_cpu__##name
+    __typeof__(type) per_cpu__##name \
+    __attribute__((section("__per_cpu")))
+
 
 /* var is in discarded region: offset to particular copy we want */
 #define per_cpu(var, cpu)  \

_______________________________________________
Xen-ppc-devel mailing list
Xen-ppc-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-ppc-devel