Re: Xen 4.14.1 on RPI4: device tree generation failed

[Tamas K Lengyel <tamas.k.lengyel@xxxxxxxxx>]

On Mon, Feb 1, 2021 at 10:23 AM Tamas K Lengyel
<tamas.k.lengyel@xxxxxxxxx> wrote:
>
> On Mon, Feb 1, 2021 at 12:54 AM Elliott Mitchell <ehem+xen@xxxxxxx> wrote:
> >
> > On Sun, Jan 31, 2021 at 10:06:21PM -0500, Tamas K Lengyel wrote:
> > > With rpi-4.19.y kernel and dtbs
> > > (cc39f1c9f82f6fe5a437836811d906c709e0661c) Xen boots fine and the
> > > previous error is not present. I get the boot log on the serial with
> > > just console=hvc0 from dom0 but the kernel ends up in a panic down the
> > > line:
> >
> > > This seems to have been caused by a monitor being attached to the HDMI
> > > port, with HDMI unplugged dom0 boots OK.
> >
> > The balance of reports seem to suggest 5.10 is the way to go if you want
> > graphics on a RP4 with Xen.  Even without Xen 4.19 is looking rickety on
> > RP4.
> >
> >
> > On Sun, Jan 31, 2021 at 09:43:13PM -0500, Tamas K Lengyel wrote:
> > > On Sun, Jan 31, 2021 at 8:59 PM Elliott Mitchell <ehem+xen@xxxxxxx> wrote:
> > > >
> > > > On Sun, Jan 31, 2021 at 06:50:36PM -0500, Tamas K Lengyel wrote:
> > > > > On Sun, Jan 31, 2021 at 6:33 PM Elliott Mitchell <ehem+undef@xxxxxxx> 
> > > > > wrote:
> > > > > > Presently the rpixen script is grabbing the RPF's 4.19 branch, dates
> > > > > > point to that last being touched last year.  Their tree is at
> > > > > > cc39f1c9f82f6fe5a437836811d906c709e0661c.
> > > > >
> > > > > I've moved the Linux branch up to 5.10 because there had been a fair
> > > > > amount of work that went into fixing Xen on RPI4, which got merged
> > > > > into 5.9 and I would like to be able to build upstream everything
> > > > > without the custom patches coming with the rpixen script repo.
> > > >
> > > > Please keep track of where your kernel source is checked out at since
> > > > there was a desire to figure out what was going on with the 
> > > > device-trees.
> > > >
> > > >
> > > > Including "console=hvc0 console=AMA0 console=ttyS0 console=tty0" in the
> > > > kernel command-line should ensure you get output from the kernel if it
> > > > manages to start (yes, Linux does support having multiple consoles at 
> > > > the
> > > > same time).
> > >
> > > No output from dom0 received even with the added console options
> > > (+earlyprintk=xen). The kernel build was from rpi-5.10.y
> > > c9226080e513181ffb3909a905e9c23b8a6e8f62. I'll check if it still boots
> > > with 4.19 next.
> >
> > So, their current HEAD.  This reads like you've got a problematic kernel
> > configuration.  What procedure are you following to generate the
> > configuration you use?
> >
> > Using their upstream as a base and then adding the configuration options
> > for Xen has worked fairly well for me (`make bcm2711_defconfig`,
> > `make menuconfig`, `make zImage`).
> >
> > Notably the options:
> > CONFIG_PARAVIRT
> > CONFIG_XEN_DOM0
> > CONFIG_XEN
> > CONFIG_XEN_BLKDEV_BACKEND
> > CONFIG_XEN_NETDEV_BACKEND
> > CONFIG_HVC_XEN
> > CONFIG_HVC_XEN_FRONTEND
> >
> > Should be set to "y".
>
> Yes, these configs are all set the same way for all Linux builds by the 
> script:
>         make O=.build-arm64 ARCH=arm64
> CROSS_COMPILE=aarch64-none-linux-gnu- bcm2711_defconfig xen.config
>
> I tried with both the rpi-5.10.y and rpi-5.9.y, neither boot up as
> dom0. So far only 4.19 boots.

rpi-5.4.y boots but ends up in yet another different kernel panic:

(XEN) d0v1: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER0
(XEN) d0v2: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER0
(XEN) d0v3: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER0
[    0.354800] Detected PIPT I-cache on CPU1
[    0.360473] Xen: initializing cpu1
[    0.360508] CPU1: Booted secondary processor 0x0000000001 [0x410fd083]
[    0.361674] Detected PIPT I-cache on CPU2
[    0.367323] Xen: initializing cpu2
[    0.367357] CPU2: Booted secondary processor 0x0000000002 [0x410fd083]
[    0.368460] Detected PIPT I-cache on CPU3
[    0.374110] Xen: initializing cpu3
[    0.374144] CPU3: Booted secondary processor 0x0000000003 [0x410fd083]
[    0.374357] smp: Brought up 1 node, 4 CPUs
[    0.421250] SMP: Total of 4 processors activated.
[    0.426051] CPU features: detected: 32-bit EL0 Support
[    0.431344] CPU features: detected: CRC32 instructions
[    0.459837] ------------[ cut here ]------------
[    0.463848] CPU: CPUs started in inconsistent modes
[    0.463925] Internal error: aarch64 BRK: f2000800 [#1] PREEMPT SMP
[    0.475143] Modules linked in:
[    0.478308] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.4.83-v8+ #1
[    0.484712] Hardware name: Raspberry Pi 4 Model B Rev 1.1 (DT)
[    0.490686] pstate: 60000005 (nZCv daif -PAN -UAO)
[    0.495615] pc : smp_cpus_done+0x74/0x98
[    0.499642] lr : smp_cpus_done+0x74/0x98
[    0.503687] sp : ffffffc01002bdf0
[    0.507102] x29: ffffffc01002bdf0 x28: 0000000000000000
[    0.512545] x27: 0000000000000000 x26: ffffffc010f08958
[    0.517989] x25: ffffffc010f286c8 x24: 0000000000000040
[    0.523442] x23: ffffffc010f08000 x22: ffffffc010f28000
[    0.528876] x21: ffffffc010f08918 x20: ffffffc010f08aa0
[    0.534331] x19: 0000000000000100 x18: 0000000000000000
[    0.539764] x17: 000000004d4bcabc x16: 00000000deadbeef
[    0.545209] x15: 0000000000000030 x14: ffffffffffffffff
[    0.550652] x13: ffffffc09002ba87 x12: ffffffc01002ba8f
[    0.556096] x11: ffffffc01002bdf0 x10: ffffffc01002bdf0
[    0.561540] x9 : 00000000ffffffc8 x8 : 636e69206e692064
[    0.566984] x7 : 6574726174732073 x6 : ffffffc010f09000
[    0.572427] x5 : ffffffc010f09098 x4 : ffffffc01002c000
[    0.577871] x3 : 0000000000000000 x2 : 0000000000000000
[    0.583315] x1 : 0000000000000000 x0 : ffffff8036948000
[    0.588759] Call trace:
[    0.591310]  smp_cpus_done+0x74/0x98
[    0.594999]  smp_init+0xe4/0xfc
[    0.598245]  kernel_init_freeable+0x134/0x27c
[    0.602726]  kernel_init+0x1c/0x118
[    0.606326]  ret_from_fork+0x10/0x18
[    0.610016] Code: 540000c0 90fff480 9125a000 97cb8545 (d4210000)
[    0.616251] ---[ end trace 828ddf3cc765197a ]---
[    0.620987] note: swapper/0[1] exited with preempt_count 1
[    0.626610] Kernel panic - not syncing: Attempted to kill init!
exitcode=0x0000000b
[    0.634425] SMP: stopping secondary CPUs
[    0.638511] ---[ end Kernel panic - not syncing: Attempted to kill
init! exitcode=0x0000000b ]---

Tamas