[Xen-devel] Bringing up sequence for non-boot CPU fails

[Oleksandr Tyshchenko <oleksandr.tyshchenko@xxxxxxxxxxxxxxx>]

Hi, all.
I am working with DRA7 (OMAP5) platform.

I rarely see that Hypervisor hangs during bringing up CPU1.
It is not a deadlock, XEN console is working, but system is unusable(

We have next situation.
The CPU0 can not exit from busy loop in __cpu_up func (smpboot.c),
he is waiting CPU1 to become online but CPU1 stucks during bring up sequence.
The last print which I see is "- Turning on paging -".
It seems, it happens during enable MMU/Cache or immediately after it.
But anyway,
instruction "mov pc, r1" does not result to transition to the label "paging".

(XEN) Latest ChangeSet: Mon Feb 17 13:43:16 2014 +0200 git:c30f01f
(XEN) Processor: 412fc0f2: "ARM Limited", variant: 0x2, part 0xc0f, rev 0x2
(XEN) 32-bit Execution:
(XEN)   Processor Features: 00001131:00011011
(XEN)     Instruction Sets: AArch32 Thumb Thumb-2 ThumbEE Jazelle
(XEN)     Extensions: GenericTimer Security
(XEN)   Debug Features: 02010555
(XEN)   Auxiliary Features: 00000000
(XEN)   Memory Model Features: 10201105 20000000 01240000 02102211
(XEN)  ISA Features: 02101110 13112111 21232041 11112131 10011142 00000000
(XEN) Platform: TI DRA7
(XEN) /psci method must be smc, but is: "hvc"
(XEN) Set AuxCoreBoot1 to 00000000dec0004c (0020004c)
(XEN) Set AuxCoreBoot0 to 0x20
(XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27
(XEN) Using generic timer at 6144 KHz
(XEN) GIC initialization:
(XEN)         gic_dist_addr=0000000048211000
(XEN)         gic_cpu_addr=0000000048212000
(XEN)         gic_hyp_addr=0000000048214000
(XEN)         gic_vcpu_addr=0000000048216000
(XEN)         gic_maintenance_irq=25
(XEN) GIC: 192 lines, 2 cpus, secure (IID 0000043b).
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Allocated console ring of 16 KiB.
(XEN) VFP implementer 0x41 architecture 4 part 0x30 variant 0xf rev 0x0
(XEN) Bringing up CPU1
- CPU 00000001 booting -
- NOT HYP, setting it ... -
- Xen starting in Hyp mode -
- Setting up control registers -
- Turning on paging -

Currently I am using XEN 4.4 RC3, but this issue is reproduced since XEN 4.3.
We do not have any patches related to bring up sequence on top of 4.4
RC3 except one:
So long as our bootloader does not set HYP mode we need to keep this
hack in Hypervisor.

diff --git a/xen/arch/arm/arm32/head.S b/xen/arch/arm/arm32/head.S
index 96230ac..6fa1e3d 100644
--- a/xen/arch/arm/arm32/head.S
+++ b/xen/arch/arm/arm32/head.S
@@ -174,6 +174,15 @@ common_start:
         teq   r0, #0x1a              /* Hyp Mode? */
         beq   hyp

+        PRINT("- NOT HYP, setting it ... -\r\n")
+        bl    enter_hyp_mode
+
+        /* Check that set HYP mode was succesfull */
+        mrs   r0, cpsr
+        and   r0, r0, #0x1f          /* Mode is in the low 5 bits of CPSR */
+        teq   r0, #0x1a              /* Hyp Mode? */
+        beq   hyp
+
         /* OK, we're boned. */
         PRINT("- Xen must be entered in NS Hyp mode -\r\n" \
               "- Please update the bootloader -\r\n")
@@ -547,6 +556,24 @@ putn:   mov   pc, lr

 #endif /* !EARLY_PRINTK */

+GLOBAL(enter_hyp_mode)
+enter_hyp_mode:
+        adr   r0, save
+        stmea r0, {r4-r13,lr}
+        ldr   r12, =0x102
+        adr   r0, hyp_return
+        dsb
+        isb
+        dmb
+        smc   #0
+hyp_return:
+        adr   r0, save
+        ldmfd r0, {r4-r13,pc}
+save:
+        .rept 11
+        .word 0
+        .endr
+
 /*
  * Local variables:
  * mode: ASM


Please, could anyone give me advice about this issue? And what do you
think about solution to exit from busy loop by timeout and restart
CPU1 bringing up sequence in this case?

Thank you.

-- 

Oleksandr Tyshchenko | Embedded Developer
GlobalLogic
www.globallogic.com

http://www.globallogic.com/email_disclaimer.txt

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel