Re: [Xen-users] ARM: Xen on Vexpress

[Jeenu Viswambharan <Jeenu.Viswambharan@xxxxxxx>]

On Wed, Jun 11, 2014 at 15:16:12, Ian Campbell wrote:
> On Wed, 2014-06-11 at 14:54 +0100, Jeenu Viswambharan wrote:
> > It turned out to be a GCC 4.6 issue where the offset of vcpu inside
> > struct domain was wrongly generated as 0xe4 while it's 0xf8. I
> > almost forgot the recommendation to use GCC 4.8 in the Wiki. It goes
> > past that with 2014.04 tool chain from Linaro. My bad!
>
> You mean the contents of asm-offset.h was wrong or something else?
>
> I routinely use Debian's gcc 4.6.3 to build Xen and I've not seen this.
> Perhaps it's an issue with your specific build of gcc?

To the call to irq_to_pending in gic_route_irq_to_guest, using GCC
4.8.3 (Linaro 2014.04), I get (*-objdump -S):

    /* TODO: do not assume delivery to vcpu0 */
    p = irq_to_pending(d->vcpu[0], irq->irq);
  24d420:       e59830f8        ldr     r3, [r8, #248]  ; 0xf8
  24d424:       e5930000        ldr     r0, [r3]
  24d428:       e5961000        ldr     r1, [r6]
  24d42c:       eb0020c4        bl      255744 <irq_to_pending>

But the same code with 4.6.3:

    /* TODO: do not assume delivery to vcpu0 */
    p = irq_to_pending(d->vcpu[0], irq->irq);
  237bb4:       e59830e4        ldr     r3, [r8, #228]  ; 0xe4  <- Wrong offset
        return -EBUSY;

    desc->action  = new;
    dsb();

    return 0;
  237bb8:       e1a05002        mov     r5, r2
        xfree(action);
        goto out;
    }

    /* TODO: do not assume delivery to vcpu0 */
    p = irq_to_pending(d->vcpu[0], irq->irq);
  237bbc:       e5961000        ldr     r1, [r6]
  237bc0:       e5930000        ldr     r0, [r3]                <- Faults here
  237bc4:       eb001926        bl      23e064 <irq_to_pending>

4.6.3 is what got installed via. apt-get on my Ubuntu 12.04 PC.
Admittedly I didn't dig further as the new tool chain saved me there.

> > Xen is now launching DOM0. But next issue I'm now facing is that
> > Linux refusing to run when loaded at 0xa0008000 - it panics and
> > complains about failing to allocate memory in
> > mm/memblock.c:memblock_alloc_base().  FWIW, I've built the kernel
> > v3.14 with vexpress_defconfig and enabled Xen and highmem support.
> > The kernel however is happy to boot when loaded at 0x80008000
> > (without Xen). The same with Xen though, the kernel reports
> > 'uncompression error'.
> >
> > It's little to do with Xen, but are there any other options required
> > to be enabled in the DOM0 Kernel to run from 0xa0008000?
>
> Your kernel needs to be relocatable. I'm not 100% which option that is
> (perhaps CONFIG_ARM_PATCH_PHYS_VIRT) but if you build a multiplatform
> kernel (CONFIG_ARCH_MULTIPLATFORM) it certainly will be.

Just found out that both of the configs are enabled in my build, and
I've no idea what's going on.

Like I said, when loaded at 0xa0008000, the kernel didn't boot neither
on bare metal nor Xen. But at 0x80008000, the kernel was happy to boot
bare metal but not with Xen. Does Xen care where the kernel is, as long
as the kernel is happy to execute from wherever it's loaded? If not,
since the latter looks more promising, I was hoping to debug that
instead of the former.

Maybe someone who had seen Xen on Vexpress comment on any special kernel
configs. Julien, perhaps?

> > Also could you tell me at what point Xen jumps to guest entry? I
> > tried break at the eret instruction following the label
> > return_to_hypervisor, but the break point doesn't seem to hit.
>
> It should, that's the right place.

Hm, OK. Good to have that confirmed.

--
Jeenu

-- IMPORTANT NOTICE: The contents of this email and any attachments are 
confidential and may also be privileged. If you are not the intended recipient, 
please notify the sender immediately and do not disclose the contents to any 
other person, use it for any purpose, or store or copy the information in any 
medium.  Thank you.

ARM Limited, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Registered 
in England & Wales, Company No:  2557590
ARM Holdings plc, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, 
Registered in England & Wales, Company No:  2548782
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users