[Top] [All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] Failed to enable MMU when booting EFI on Seattle

On 10/7/2014 4:21 AM, Ian Campbell wrote:

On Mon, 2014-10-06 at 22:29 -0500, Suravee Suthikulpanit wrote:

Ian/Roy,

So, I have found that in arch/arm/arm64/head.S, after we enable MMU and
execute "br x1" (where x1 = 0x2005F8, the VA of "paging:"), it went to
address 0x2005F8, which has all zeros value. This is why we getting the
"Synchronous Exception"

......
1:
          PRINT("- Turning on paging -\r\n")

          ldr   x1, =paging            /* Explicit vaddr, not RIP-relative */
          mrs   x0, SCTLR_EL2
          orr   x0, x0, #SCTLR_M       /* Enable MMU */
          orr   x0, x0, #SCTLR_C       /* Enable D-cache */
          dsb   sy                     /* Flush PTE writes and finish
reads */
        b .
          msr   SCTLR_EL2, x0          /* now paging is enabled */
          isb                          /* Now, flush the icache */
          br    x1                     /* Get a proper vaddr into PC */
paging:
.....

Notes:
* This is not the case when booting non-EFI.

* If I do "add x1, x1, x20", which puts the PA of paging into x1, it
seems to branch to "paging:" fine. Although, I don't this this is the
right thing to do since it should already be using VA).

So, I inspected the page tables and compare the ones from booting with
and without EFI. At this point, it seems that the contents of the page
tables are set up properly. The only thing different is the phys-offset
and the location of the page table:

BOOTING WITH EFI:

        x20 (phys-offset) = 0x83fc2d0000
        x19 (start paddr) = 0x83fc4d0000
        (.text starts at 2MB = 0x200000 offset)

        boot_pgtable (pa) = x20 + 0x2f2000 = 0x83fc5c2000
        boot_first (pa)   = x20 + 0x2f3000 = 0x83fc5c3000
        boot_second(pa)   = x20 + 0x2f5000 = 0x83fc5c5000
        boot_third (pa)   = x20 + 0x2f6000 = 0x83fc5c6000

BOOTING WITHOUT EFI:

        x20 (phys-offset) = 0x8008500000
        x19 (start paddr) = 0x8008700000
        (.text starts at 2MB = 0x200000 offset)

        boot_pgtable (pa) = x20 + 0x2f2000 = 0x80087f2000
        boot_first (pa)   = x20 + 0x2f3000 = 0x80087f3000
        boot_second(pa)   = x20 + 0x2f5000 = 0x80087f5000
        boot_third (pa)   = x20 + 0x2f6000 = 0x80087f6000

Here is the memory map available from UEFI:
        
Shell> memmap
Type      Start            End              #pages             Attributes
RT_Data   0000008000000000-0000008000FFFFFF 0000000000001000
800000000000000F
Available 0000008001000000-0000008007FFDFFF 0000000000006FFE
000000000000000F
BS_Code   0000008007FFE000-0000008007FFFFFF 0000000000000002
000000000000000F
Available 0000008008000000-000000801FFFDFFF 0000000000017FFE
000000000000000F
BS_Data   000000801FFFE000-000000801FFFFFFF 0000000000000002
000000000000000F
Available 0000008020000000-000000802FFFFFFF 0000000000010000
000000000000000F
RT_Code   0000008030000000-0000008030000FFF 0000000000000001
800000000000000F
Available 0000008030001000-00000083F0FFFFFF 00000000003C0FFF
000000000000000F
BS_Data   00000083F1000000-00000083F101FFFF 0000000000000020
000000000000000F
Available 00000083F1020000-00000083FC5D2FFF 000000000000B5B3
000000000000000F
LoaderCode 00000083FC5D3000-00000083FC6B1FFF 00000000000000DF
000000000000000F

However, it seems that the location falls in the "Available", which
should be ok to use.


Right, I don't see anything wrong with the location of the page tables.

What do the contents of these PTs look like?

In particular for Xen's load address of 0x200000 I think entries at
boot_pgtable[0], boot_first[0], boot_second[0] and boot_third[0x20] are
of interest. Do they correctly map their next level and/or the physical
address with the actual Xen bits in?

For completeness it would also be worth knowing the entries
corresponding to the 1:1 pa mapping.

For phys-offset 0x8008500000 those would be offsets [0x1, 0x0, 0x42,
0x100] (I think, but please do check my maths!). I think this will
result in boot_pgtable[1] mapping boot_first_id which will contain a
gigabyte page super mapping in slot [0].

Phys-offset 0x83fc2d0000 ought to be offsets [0x1, 0xf, 0x1e1, 0xd].
Which I would again expect to result in boot_pgtable[1] mapping
boot_first_id with a gigabyte mapping in slot 0xf this time.

I expect the PA mapping to be correct, since as you say we are able to
run off it, it's worth checking though in case we are somehow creating
VA and PA-1:1 mappings which conflict (e.g. writing boot_first instead
of boot_first_id when creating the PA mappings -- that sort of thing).

Ian.


So, I have experimented with:

        /*
         * Preserve x0 (fdt pointer) across call to __flush_dcache_area,
         * restore for entry into Xen.
         */
        mov   x20, x0

        /*
         * Flush dcache covering current runtime addresses
         * of xen text/data. Then flush all of icache.
         */
        adrp  x1, _start
        add   x1, x1, #:lo12:_start
        mov   x0, x1    <--- Fixed X0
        adrp  x2, _end
        add   x2, x2, #:lo12:_end
        sub   x1, x2, x1

        bl    __flush_dcache_area
        ic    ialluis
        tlbi  alle2     <---- Added this
Now, that I added the "tlbi alle2", it works after we enable MMU :) I guess even though we set up the new page table, but it was still using the stale info in the TLB. 

On to the next hurdle:

Shell> FS0:xen -cfg=xen-seattle.cfg
Xen 4.5-unstable (c/s Mon Oct 6 10:16:52 2014 -0500 git:f0ffd29-dirty) EFI loader 
Image: 0x00000083fbbca000-0x00000083fc4ca890
- UART enabled -
- CPU 00000000 booting -
- Current EL 00000008 -
- Xen starting at EL2 -
- Zero BSS -
- Setting up control registers -
- Setup boot first -
- Setup boot second -
- Setup boot third -
- Turning on paging -
- Ready -
(XEN) Checking for initrd in /chosen
(XEN) RAM: 0000008001000000 - 0000008007ffdfff
(XEN) RAM: 0000008007ffe000 - 0000008007ffffff
(XEN) RAM: 0000008008000000 - 000000801fffdfff
(XEN) RAM: 000000801fffe000 - 000000801fffffff
(XEN) RAM: 0000008020000000 - 000000802fffffff
(XEN) RAM: 0000008030001000 - 00000083f0ffffff
(XEN) RAM: 00000083f1000000 - 00000083f101ffff
(XEN) RAM: 00000083f1020000 - 00000083fbbc7fff
(XEN) RAM: 00000083fc4ce000 - 00000083fc4cefff
(XEN) RAM: 00000083fc6b2000 - 00000083fec25fff
(XEN) RAM: 00000083fec26000 - 00000083fee8bfff
(XEN) RAM: 00000083fee8c000 - 00000083ff225fff
(XEN) RAM: 00000083ff226000 - 00000083ff263fff
(XEN) RAM: 00000083ff265000 - 00000083ff2c4fff
(XEN) RAM: 00000083ffe70000 - 00000083ffffffff
(XEN)
(XEN) MODULE[0]: 00000083fc4cb000 - 00000083fc4ce000 Device Tree
(XEN) MODULE[1]: 00000083fbbca000 - 00000083fc4ca890 Kernel console=hvc0 console=ttyAMA0,115200 earlycon=pl011,0xe1010000 show_styx_info root=/dev/sda2 rootwait maxcpus=1 
(XEN) MODULE[2]: 0000008020000000 - 00000080209e6950 Kernel
(XEN)
(XEN) Command line: FS0:xen no-bootscrub console=dtuart conswitch=x dtuart=serial0 noreboot sync_console dom0_mem=256M dom0_max_vcpus=2 
(XEN) Placing Xen at 0x000000802fe00000-0x0000008030000000
(XEN) Update BOOTMOD_XEN from 00000083fc4d0000-00000083fc5d2d81 => 000000802fe00000-000000802ff02d81 
(XEN) Hypervisor Trap. HSR=0x96000044 EC=0x25 IL=1 Syndrome=0x44
(XEN) CPU0: Unexpected Trap: Hypervisor
(XEN) ----[ Xen-4.5-unstable  arm64  debug=y  Not tainted ]----
(XEN) CPU:    0
(XEN) PC:     00000000002794b4 bootmem_region_add+0x194/0x1c4
(XEN) LR:     00000000002794ac
(XEN) SP:     00000000002afd50
(XEN) CPSR:   800003c9 MODE:64-bit EL2h (Hypervisor, handler)
(XEN)      X0: 0000800000000000  X1: 0000800000000000  X2: 0000000000000000
(XEN)      X3: 0000800000000000  X4: ffffffffffffffff  X5: 0000000000000000
(XEN)      X6: 0000800000000010  X7: 000000000000000a  X8: 0000000000000000
(XEN)      X9: 0000000000000010 X10: 00000000002afbb8 X11: 0000000000000038
(XEN)     X12: 000000000000000a X13: 000000000025d380 X14: 0000000000000030
(XEN)     X15: 0000000000400000 X16: 0000000000000000 X17: 0000000000287fdc
(XEN)     X18: 000000802feea000 X19: 0000000008001001 X20: 0000000000290048
(XEN)     X21: 0000000008007ffe X22: 0000000000000000 X23: 0000008007ffe000
(XEN)     X24: 00000000002794e4 X25: 00000000002794e4 X26: ffffffffffffffff
(XEN)     X27: 0000000000298038 X28: 0000008007ffe000  FP: 00000000002afd50
(XEN)
(XEN)   VTCR_EL2: 80000000
(XEN)  VTTBR_EL2: 0000000000000000
(XEN)
(XEN)  SCTLR_EL2: 30cd183d
(XEN)    HCR_EL2: 000000000038643f
(XEN)  TTBR0_EL2: 000000802feec000
(XEN)
(XEN)    ESR_EL2: 96000044
(XEN)  HPFAR_EL2: 0000000000000000
(XEN)    FAR_EL2: 0000800000000000
(XEN)
(XEN) Xen stack trace from sp=00000000002afd50:
(XEN)    00000000002afd80 0000000000279530 0000000000290048 0000000000000000
(XEN)    0000008001000000 0000008001000000 00000000002afdd0 000000000024b154
(XEN)    0000000000000000 0000000000000000 0000008001000000 0000008001000000
(XEN)    0000008007ffe000 00000000002794e4 00000000002afe20 00000000002834ac
(XEN)    00000000002afe20 0000000000283d50 0000000000298030 0000000000000418
(XEN)    0000008001000000 0000008007ffe000 0000008007ffe000 0000000000000000
(XEN)    0000000000298038 000000000fffffff 00000083fff702b0 0000000000200690
(XEN)    00000083fc4d0000 00000083fc2d0000 00000083fc4cb000 0000000000000000
(XEN)    0000000000400000 0000000000000000 0000000000000001 00000083fc544be0
(XEN)    00000083fc5800f0 00000083fc5800e0 0000000000000000 0000000000003000
(XEN)    00000083fc4cb000 0000000006ffe000 0000000000000000 0000008001000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000
(XEN) Xen call trace:
(XEN)    [<00000000002794b4>] bootmem_region_add+0x194/0x1c4 (PC)
(XEN)    [<00000000002794ac>] bootmem_region_add+0x18c/0x1c4 (LR)
(XEN)    [<0000000000279530>] init_boot_pages+0x4c/0x174
(XEN)    [<000000000024b154>] dt_unreserved_regions+0xbc/0xd0
(XEN)    [<0000000000283d50>] start_xen+0xc38/0xc58
(XEN)    [<0000000000200690>] paging+0x88/0xc0
(XEN)
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) CPU0: Unexpected Trap: Hypervisor
(XEN)
(XEN) ****************************************
(XEN)
(XEN) Manual reset required ('noreboot' specified)

Any thoughts??

Thanks,

Suravee





_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel

 

©2013 Xen Project, A Linux Foundation Collaborative Project. All Rights Reserved.
Linux Foundation is a registered trademark of The Linux Foundation.
Xen Project is a trademark of The Linux Foundation.

Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.