Re: [Xen-users] Issue in booting linux DomU in arm hw

[André Przywara <andre.przywara@xxxxxxx>]

On 03/12/2019 21:01, Julien Grall wrote:

Hi,

> Thank you for the details but please avoid to drop the ML from the CC List.

thanks Julien for looping me in.

> 
> On 24/11/2019 10:21, dharani kumar wrote:
>> Dear Julien,
>>    - Where Xen is loaded
>> Xen is present in the RAM at 0x42000000. Load address and entry point
>> given in the uImage is 0x40200000.
>> Size of Xen 853328bytes.
>>      - Memory banks
>> I dont find any mention of memory bank in the processor user manual.
> 
> They should be dumped by Xen when earlyprintk are enabled. I think
> CONFIG_EARLY_PRINTK=sun7i should do the job for you.
> 
>>      - Any reserved regions
>> Memory mapping page says 0x40000000 to 0xFFFFFFFF is meant for SDRAM.
>>      - Which exact version of the pine64 are you using?
>> I am using PineA64+ (1GB RAM Variant)
>>      - What's your device-tree?
>> Device tree is built from mainline kernel. sun50i-a64-pine64-plus.dtb
> 
> I have CCed Andre who IIRC has a pine64+. @Andre, do you know whether
> there is anything specific do to on the board to get Xen booting?
> 
> It looks like Xen will crash at boot (re-pasting the log below as the
> e-mail wasn't on xen-users):
> 
> [mmc]: MMC Device 2 not found
> [mmc]: mmc 2 not find, so not exit
> INFO:    BL3-1: Next image address = 0x40200000
> INFO:    BL3-1: Next image spsr = 0x3c9

So this smells like you are using an Allwinner provided firmware, which
is not only ancient and broken in various ways, but also not supported.
I think the particular problem at hand is that ATF runs in the first MBs
of DRAM. IIRC they had some /memreserve/ in *their* DT to cover this,
but the mainline DT certainly hasn't.
Anyway, these days there is very little reason to use this outdated
firmware, especially when it comes to mainline software.
So please follow the instructions in U-Boot [1] to build the latest
firmware from mainline U-Boot/ATF sources.

If you are really desperate, you could also use some prebuilt firmware
packages, even though they are already old as well:
https://github.com/apritzel/pine64

I just tried this both with latest upstream and the old image, using
your U-Boot commands below. Just use "booti" instead of "bootm". I got
far into the Dom0 kernel. With the right DT (I used an older one) you
might as well reach the prompt.

Cheers,
Andre.

[1]
https://gitlab.denx.de/u-boot/u-boot/blob/master/board/sunxi/README.sunxi64

>  Xen 4.12.1
> (XEN) Xen version 4.12.1 (root@) (gcc (Ubuntu/Linaro
> 5.4.0-6ubuntu1~16.04.12) 5.4.0 20160609) debug=n  Thu Nov 21 20:03:32
> UTC 2019
> (XEN) Latest ChangeSet:
> (XEN) Processor: 410fd034: "ARM Limited", variant: 0x0, part 0xd03, rev 0x4
> (XEN) 64-bit Execution:
> (XEN)   Processor Features: 0000000000002222 0000000000000000
> (XEN)     Exception Levels: EL3:64+32 EL2:64+32 EL1:64+32 EL0:64+32
> (XEN)     Extensions: FloatingPoint AdvancedSIMD
> (XEN)   Debug Features: 0000000010305106 0000000000000000
> (XEN)   Auxiliary Features: 0000000000000000 0000000000000000
> (XEN)   Memory Model Features: 0000000000001122 0000000000000000
> (XEN)   ISA Features:  0000000000011120 0000000000000000
> (XEN) 32-bit Execution:
> (XEN)   Processor Features: 00000131:00011011
> (XEN)     Instruction Sets: AArch32 A32 Thumb Thumb-2 Jazelle
> (XEN)     Extensions: GenericTimer Security
> (XEN)   Debug Features: 03010066
> (XEN)   Auxiliary Features: 00000000
> (XEN)   Memory Model Features: 10201105 40000000 01260000 02102211
> (XEN)  ISA Features: 02101110 13112111 21232042 01112131 00011142 00011121
> (XEN) Generic Timerder IRQ: phys=30 hyp=26 virt=27 Freq: 24000 KHz
> (XEN) GICv2 initialization:
> (XEN)         gic_dist_addr=0000000001c81000
> (XEN)         gic_cpu_addr=0000000001c82000
> (XEN)         gic_hyp_addr=0000000001c84000
> (XEN)         gic_vcpu_addr=0000000001c86000
> (XEN)         gic_maintenance_irq=25
> (XEN) GICv2: 224 lines, 4 cpus, secure (IID 0200143b).
> (XEN) XSM Framework v1.0.0 initialized
> (XEN) Initialising XSM SILO mode
> (XEN) Using scheduler: SMP Credit Scheduler rev2 (credit2)
> (XEN) Initializing Credit2 scheduler
> (XEN) Allocated console ring of 16 KiB.
> (XEN) Bringing up CPU1
> (XEN) Bringing up CPU2
> (XEN) Bringing up CPU3
> (XEN) Brought up 4 CPUs
> (XEN) P2M: 40-bit IPA with 40-bit PA and 8-bit VMID
> (XEN) P2M: 3 levels with order-1 root, VTCR 0x80023558
> (XEN) I/O virtualisation disabled
> (XEN) *** LOADING DOMAIN 0 ***
> (XEN) Loading Domd0 kernel from boot module @ 0000000052000000
> (XEN) Allocating 1:1 mappings totalling 256MB for dom0:
> (XEN) BANK[0] 0x00000060000000-0x00000070000000 (256MB)
> (XEN) Grant table range: 0x00000040200000-0x00000040240000
> (XEN) Allocating PPI 16 for event channel interrupt
> (XEN) Loading zImage from 0000000052000000 to
> 0000000060080000-0000000061971200
> (XEN) Loading dom0 DTB to 0x0000000068000000-0x00000000680056c9
> (XEN) Initial low memory virq threshold set at 0x4000 pages.
> (XEN) Scrubbing Free RAM in background
> (XEN) Std. Loglevel: Errors and warnings
> (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
> (XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input)
> (XEN) Data Abort Trap. Syndrome=0x5
> (XEN) Walking Hypervisor VA 0x7fffd0d98 on CPU0 via TTBR 0x0000000040308000
> (XEN) 0TH[0x0] = 0x000000004030ff7f
> (XEN) 1ST[0x1f] = 0x0000000000000000
> (XEN) CPU0: Unexpected Trap: Data Abort
> (XEN) ----[ Xen-4.12.1  arm64  debug=n   Not tainted ]----
> (XEN) CPU:    0
> (XEN) PC:     00000000002186d8 page_alloc.c#free_heap_pages+0x38/0x518
> (XEN) LR:     000000000021a68c
> (XEN) SP:     000080003fff7d80
> (XEN) CPSR:   20000249 MODE:64-bit EL2h (Hypervisor, handler)
> (XEN)      X0: 00000007fffd0d88  X1: 0000000000000000  X2: 0000000000000000
> (XEN)      X3: ffffffffffffffff  X4: 0000000000000000  X5: 0000000000041000
> (XEN)      X6: 00000007fffd0d88  X7: 0000000000000001  X8: 00000000000000d8
> (XEN)      X9: 0000000000306000 X10: 000000000026c828 X11: 00000000002515e0
> (XEN)     X12: 0000000000014400 X13: 0000000000000001 X14: 0000000000000000
> (XEN)     X15: 0000000000286000 X16: 0000000000000000 X17: 0000000000000000
> (XEN)     X18: 0000000000286000 X19: 0000000000000051 X20: 0000000000286000
> (XEN)     X21: 0000000000286000 X22: 0000000000286000 X23: 0000000000286000
> (XEN)     X24: 0000000000000000 X25: 00000000002d8000 X26: 0000000000286000
> (XEN)     X27: 0000000000000000 X28: 00000007fffd0d88  FP: 0000000000000000
> (XEN)
> (XEN)   VTCR_EL2: 80023558
> (XEN)  VTTBR_EL2: 00004cb9f4e3b990
> (XEN)
> (XEN)  SCTLR_EL2: 30cd183d
> (XEN)    HCR_EL2: 000000000000003a
> (XEN)  TTBR0_EL2: 0000000040308000
> (XEN)
> (XEN)    ESR_EL2: 96000005
> (XEN)  HPFAR_EL2: 00000000b7a551e0
> (XEN)    FAR_EL2: 00000007fffd0d98
> (XEN)
> (XEN) Xen stack trace from sp=000080003fff7d80:
> (XEN)    0000000000000051 00000007fffd0d88 0000000000000000
> 0000000000286378
> (XEN)    0000000000286000 0000000000000001 00000000002d8000
> 0000000000286000
> (XEN)    0000000000286000 0000000000286000 000000000021a68c
> 00000000002863a0
> (XEN)    0000000000297ed8 d37edb4bfbffdfff 0000000000287000
> 0000000000051000
> (XEN)    0000000040287000 0000000040287000 0000000000316000
> 0000000000000004
> (XEN)    0000000000280780 0000000000316448 0000000000000004
> 00000000002b0070
> (XEN)    0000000000252030 00000000002d8000 0000000000285280
> 00000000002863a0
> (XEN)    0000000000316000 0000000000000004 000080001801e000
> 00000000002d8000
> (XEN)    0000000000285280 0000000000316458 00000000002558f0
> ffe7ffffff9edfbb
> (XEN)    0000000000298e94 fc796dda7fffffff 0000000000000000
> 0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> (XEN) Xen call trace:
> (XEN)    [<00000000002186d8>] page_alloc.c#free_heap_pages+0x38/0x518 (PC)
> (XEN)    [<000000000021a68c>] page_alloc.c#init_heap_pages+0xf4/0x470 (LR)
> (XEN)
> (XEN)
> (XEN) ****************************************
> (XEN) Panic on CPU 0:
> (XEN) CPU0: Unexpected Trap: Data Abort
> (XEN) ****************************************
> 
> 
> 
>>
>> Allwinner A64 user manual:
>> http://files.pine64.org/doc/datasheet/pine64/Allwinner_A64_User_Manual_V1.0.pdf
>>
>>
>> My boot script:
>> setenv xen_addr_r 0x42000000
>> setenv kernel_addr_r 0x52000000
>> setenv kernel_size 0x18F1200
>> setenv dtb_addr_r 0x58000000
>>
>> setenv xen_bootargs 'console=dtuart dtuart=/soc/serial@1c28000
>> dom0_max_vcpus=2 dom0_mem=256M'
>> setenv dom0_bootargs 'console=hvc0 earlyprintk=xen ignore_loglevel
>> clk_ignore_unused root=/dev/mmcblk0p2 rw rootwait panic=10'
>>
>> ext4load mmc 0:2 $kernel_addr_r /boot/Image
>> ext4load mmc 0:2 $xen_addr_r /boot/xen-uImage
>> ext4load mmc 0:2 $dtb_addr_r
>> /boot/dts/allwinner/sun50i-a64-pine64-plus.dtb
>>
>> fdt addr $dtb_addr_r
>> fdt resize
>> fdt set /chosen xen,xen-bootargs \"$xen_bootargs\"
>> fdt resize
>> fdt set /chosen xen,dom0-bootargs \"$dom0_bootargs\"
>> fdt mknode /chosen modules
>> fdt set /chosen/modules '#address-cells' <1>
>> fdt set /chosen/modules '#size-cells' <1>
>> fdt mknode /chosen/modules module@0
>> fdt set /chosen/modules/module@0 compatible xen,linux-zimage
>> xen,multiboot-module
>> fdt set /chosen/modules/module@0 reg <$kernel_addr_r $kernel_size>
>> bootm $xen_addr_r - $dtb_addr_r
>>
>> Regards,
>> Dharani.
>>
>>
>> On Sat, Nov 23, 2019 at 11:18 PM Julien Grall <julien@xxxxxxx> wrote:
>>>
>>> On 21/11/2019 20:31, dharani kumar wrote:
>>>> Dear Julien,
>>>
>>> Hello,
>>>
>>>> I took your advise and stopped using Xen 4.6.5.
>>>> I downloaded Xen RELEASE4.12.1 and built it native on Pine64.
>>>> When i boot xen, xen traps while starting dom0.
>>>> I used the same kernel image with xen 4.6.5 before, it ran dom0 without
>>>> any problem.
>>>> Could you help to understand the error? Thanks.
>>>
>>> It looks like Xen is trying to a page to the allocator that is not
>>> valid. Could you give a bit more information your setup such as:
>>>      - Where Xen is loaded
>>>      - Memory banks
>>>      - Any reserved regions
>>>      - Which exact version of the pine64 are you using?
>>>      - What's your device-tree?
>>>
>>> Best reagrds,
>>>
>>> -- 
>>> Julien Grall
> 


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-users