Xen project Mailing List

Hi Bertrand,

Thanks so much for the quick response!

I should have mentioned previously that this device tree and kernel Image (5.15.16) does boot properly with the rootfs without XEN. The interrupt errors are only present when booting with XEN.

These are custom boards and they do have usb c, however we are unable to boot from usb as it doesn't receive power. We currently only use the usb to flash u-boot to spi which requires us to use a separate power adapter. These boards get power and networking through a custom backplane and interface with the backplane via a sata style justice (I'm not sure what the proper terminology would be here).

Since I cannot boot to rootfs with XEN, I'm unable to show the device tree there. The only bit that I added to accommodate XEN is this bit here:

chosen {
stdout-path = "serial2:1500000n8";
bootargs = "hmp-unsafe=true";
xen,dom0-bootargs = "console=hvc0 earlyprintk=xen clk_ignore_unused root=/dev/nvme0n1p2 rw init=/sbin/init rootwait rootfstype=ext4 ignore_loglevel";
modules {
#address-cells = <2>;
#size-cells = <2>;

module@1 {
compatible = "xen,linux-zimage", "xen,multiboot-module";
reg = <0x0 0x03F80000 0x0 0x01aa8008>;
};
module@2 {
compatible = "xen,linux-initrd", "xen,multiboot-module";
reg = <0x0 0x06000000 0x0 0x02000000>;
};
};

};

The gic declaration is in the vanilla rk3399.dtsi file and hasn't been modified:

gic: interrupt-controller@fee00000 {
compatible = "arm,gic-v3";
#interrupt-cells = <4>;
#address-cells = <2>;
#size-cells = <2>;
ranges;
interrupt-controller;

reg = <0x0 0xfee00000 0 0x10000>, /* GICD */
<0x0 0xfef00000 0 0xc0000>, /* GICR */
<0x0 0xfff00000 0 0x10000>, /* GICC */
<0x0 0xfff10000 0 0x10000>, /* GICH */
<0x0 0xfff20000 0 0x10000>; /* GICV */
interrupts = <GIC_PPI 9 IRQ_TYPE_LEVEL_HIGH 0>;
its: interrupt-controller@fee20000 {
compatible = "arm,gic-v3-its";
msi-controller;
#msi-cells = <1>;
reg = <0x0 0xfee20000 0x0 0x20000>;
};

ppi-partitions {
ppi_cluster0: interrupt-partition-0 {
affinity = <&cpu_l0 &cpu_l1 &cpu_l2 &cpu_l3>;
};

ppi_cluster1: interrupt-partition-1 {
affinity = <&cpu_b0 &cpu_b1>;
};
};
};

I've attached the device tree and include files just in case you'd like to take a look at those.

Thanks so much for your help! I've been at this for weeks with very little progress.

Cheers,

Brad

On Tue, Jul 12, 2022 at 10:05 AM Bertrand Marquis <Bertrand.Marquis@xxxxxxx> wrote:

Hi Brad,

> On 11 Jul 2022, at 19:38, Brad Churchwell <brad@xxxxxxxxxxxx> wrote:
>
> Hello,
>
> I've been trying to get Xen to boot dom0 with my kernel for weeks on an rk3399 based board and thought I'd reach out for help. It looks like either Xen is not properly recreating the device tree or the interrupt controller is just failing. The hypervisor boots but falls to initramfs because it cannot find the root device (nvme on pcie). Any help would be greatly appreciated. Here is the complete boot log

>From the logs you have an issue with the interrupt controller and the fact that your NVME is behind PCIE and as such depends on ITS is probably not helping.

I would suggest to try to boot on usb as root fs for a try.

Also it could be useful to compare the device tree on xen and without xen to understand what is going on (using /proc/device-tree).

Xen seems to be ok but Linux is not happy with interrupts and is showing several issues around this area on your logs.
Could you show us an extract of your device tree around the gic declaration ?

Cheers
Bertrand

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

Re: Xen on rk3399