Need help on a issue (Unable to schedule guest for Xen on Arm)

[Ayan Kumar Halder <ayankuma@xxxxxxx>]

Hi Xen experts,

I need some pointers on an issue I am facing.

I am running my downstream port of Xen on Cortex-R52 hardware. The 
hardware consist of two R52 cores (the second core is in lockstep mode). 
So, currently the hardware does not support SMP.
The issue is that Xen is unable to schedule a guest. So 
leave_hypervisor_to_guest() ---> check_for_pcpu_work() and this does not 
return.
Debugging this, I see  check_for_pcpu_work() --> do_softirq() --> 
__do_softirq() --> timer_softirq_action().
In timer_softirq_action(), the problem I see is that for all the timers, 
"((t = heap[1])->expires < now)" is true.
    while ( (heap_metadata(heap)->size != 0) &&
            ((t = heap[1])->expires < now) )
    {
        remove_from_heap(heap, t); <<<<------ So, this gets invoked for 
all the timers.
        execute_timer(ts, t);
    }

So, further below reprogram_timer() gets invoked with timeout = 0 and it 
disables the timer. So, timer_interrupt() is never invoked.
Can someone give any pointers on what the underlying issue could be and 
how to debug further ?
I do not observe this behavior while running on R52 FVP. The difference 
is that for most of the timers "((t = heap[1])->expires < now)" is 
false, so reprogram_timer() gets invoked with non zero timeout and 
subsequently, the timer_interrupt() get invoked.
Also, looking at 
https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/arm/time.c;h=0b482d7db30c89fc70b191722a89dea8a675c2b6;hb=refs/heads/staging#l211
208 
<https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/arm/time.c;h=0b482d7db30c89fc70b191722a89dea8a675c2b6;hb=refs/heads/staging#l208> 
/* Set the timer to wake us up at a particular time.
209 
<https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/arm/time.c;h=0b482d7db30c89fc70b191722a89dea8a675c2b6;hb=refs/heads/staging#l209> 
 * Timeout is a Xen system time (nanoseconds since boot); 0 disables the timer.
210 
<https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/arm/time.c;h=0b482d7db30c89fc70b191722a89dea8a675c2b6;hb=refs/heads/staging#l210> 
 * Returns 1 on success; 0 if the timeout is too soon or is in the past. */
211 
<https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/arm/time.c;h=0b482d7db30c89fc70b191722a89dea8a675c2b6;hb=refs/heads/staging#l211> 
int reprogram_timer(s_time_t timeout)
212 
<https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/arm/time.c;h=0b482d7db30c89fc70b191722a89dea8a675c2b6;hb=refs/heads/staging#l212> 
{
213 
<https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/arm/time.c;h=0b482d7db30c89fc70b191722a89dea8a675c2b6;hb=refs/heads/staging#l213> 
    uint64_t deadline;
214 
<https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/arm/time.c;h=0b482d7db30c89fc70b191722a89dea8a675c2b6;hb=refs/heads/staging#l214> 

215 
<https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/arm/time.c;h=0b482d7db30c89fc70b191722a89dea8a675c2b6;hb=refs/heads/staging#l215> 
    if ( timeout == 0 )
216 
<https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/arm/time.c;h=0b482d7db30c89fc70b191722a89dea8a675c2b6;hb=refs/heads/staging#l216> 
    {
217 
<https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/arm/time.c;h=0b482d7db30c89fc70b191722a89dea8a675c2b6;hb=refs/heads/staging#l217> 
        WRITE_SYSREG(0, CNTHP_CTL_EL2);
218 
<https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/arm/time.c;h=0b482d7db30c89fc70b191722a89dea8a675c2b6;hb=refs/heads/staging#l218> 
        return 1; <<<<<<<<<<<<<<<<<<<<<<-------------- Shouldn't this 
be 0 as the comment suggets ?
219 
<https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/arm/time.c;h=0b482d7db30c89fc70b191722a89dea8a675c2b6;hb=refs/heads/staging#l219> 
    }
220 
<https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/arm/time.c;h=0b482d7db30c89fc70b191722a89dea8a675c2b6;hb=refs/heads/staging#l220> 

221 
<https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/arm/time.c;h=0b482d7db30c89fc70b191722a89dea8a675c2b6;hb=refs/heads/staging#l221> 
    deadline = ns_to_ticks(timeout) + boot_count;
222 
<https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/arm/time.c;h=0b482d7db30c89fc70b191722a89dea8a675c2b6;hb=refs/heads/staging#l222> 
    WRITE_SYSREG64(deadline, CNTHP_CVAL_EL2);
223 
<https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/arm/time.c;h=0b482d7db30c89fc70b191722a89dea8a675c2b6;hb=refs/heads/staging#l223> 
    WRITE_SYSREG(CNTx_CTL_ENABLE, CNTHP_CTL_EL2);
224 
<https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/arm/time.c;h=0b482d7db30c89fc70b191722a89dea8a675c2b6;hb=refs/heads/staging#l224> 
    isb();
225 
<https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/arm/time.c;h=0b482d7db30c89fc70b191722a89dea8a675c2b6;hb=refs/heads/staging#l225> 

226 
<https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/arm/time.c;h=0b482d7db30c89fc70b191722a89dea8a675c2b6;hb=refs/heads/staging#l226> 
    /* No need to check for timers in the past; the Generic Timer fires
227 
<https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/arm/time.c;h=0b482d7db30c89fc70b191722a89dea8a675c2b6;hb=refs/heads/staging#l227> 
     * on a signed 63-bit comparison. */
228 
<https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/arm/time.c;h=0b482d7db30c89fc70b191722a89dea8a675c2b6;hb=refs/heads/staging#l228> 
    return 1;
229 
<https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/arm/time.c;h=0b482d7db30c89fc70b191722a89dea8a675c2b6;hb=refs/heads/staging#l229> 
}
Kind regards,

Ayan