[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Xen on rk3399
Hi Bertrand,
I believe I understand, but just to clarify, should I leave the ppi-partitions block in rk3399.dtsi as is and disable the little cores, or should I also modify that block?
Brad
Hi Brad,
> On 12 Jul 2022, at 16:59, Brad Churchwell <brad@xxxxxxxxxxxx> wrote:
>
> Hi Bertrand,
>
> Thanks so much for the quick response!
>
> I should have mentioned previously that this device tree and kernel Image (5.15.16) does boot properly with the rootfs without XEN. The interrupt errors are only present when booting with XEN.
>
> These are custom boards and they do have usb c, however we are unable to boot from usb as it doesn't receive power. We currently only use the usb to flash u-boot to spi which requires us to use a separate power adapter. These boards get power and networking through a custom backplane and interface with the backplane via a sata style justice (I'm not sure what the proper terminology would be here).
>
> Since I cannot boot to rootfs with XEN, I'm unable to show the device tree there. The only bit that I added to accommodate XEN is this bit here:
>
> chosen {
> stdout-path = "serial2:1500000n8";
> bootargs = "hmp-unsafe=true";
> xen,dom0-bootargs = "console=hvc0 earlyprintk=xen clk_ignore_unused root=/dev/nvme0n1p2 rw init=/sbin/init rootwait rootfstype=ext4 ignore_loglevel";
> modules {
> #address-cells = <2>;
> #size-cells = <2>;
>
> module@1 {
> compatible = "xen,linux-zimage", "xen,multiboot-module";
> reg = <0x0 0x03F80000 0x0 0x01aa8008>;
> };
> module@2 {
> compatible = "xen,linux-initrd", "xen,multiboot-module";
> reg = <0x0 0x06000000 0x0 0x02000000>;
> };
> };
>
> };
>
> The gic declaration is in the vanilla rk3399.dtsi file and hasn't been modified:
>
> gic: interrupt-controller@fee00000 {
> compatible = "arm,gic-v3";
> #interrupt-cells = <4>;
> #address-cells = <2>;
> #size-cells = <2>;
> ranges;
> interrupt-controller;
>
> reg = <0x0 0xfee00000 0 0x10000>, /* GICD */
> <0x0 0xfef00000 0 0xc0000>, /* GICR */
> <0x0 0xfff00000 0 0x10000>, /* GICC */
> <0x0 0xfff10000 0 0x10000>, /* GICH */
> <0x0 0xfff20000 0 0x10000>; /* GICV */
> interrupts = <GIC_PPI 9 IRQ_TYPE_LEVEL_HIGH 0>;
> its: interrupt-controller@fee20000 {
> compatible = "arm,gic-v3-its";
> msi-controller;
> #msi-cells = <1>;
> reg = <0x0 0xfee20000 0x0 0x20000>;
> };
>
> ppi-partitions {
> ppi_cluster0: interrupt-partition-0 {
> affinity = <&cpu_l0 &cpu_l1 &cpu_l2 &cpu_l3>;
> };
>
> ppi_cluster1: interrupt-partition-1 {
> affinity = <&cpu_b0 &cpu_b1>;
> };
> };
Xen is not supporting PPI partitions.
I also saw that your booting using SMP unsafe so all core are enabled by Xen.
Right now big/little is not supported by Xen and here you might have some interrupts in your system which can only occur on some specific cores.
I would suggest to try the following:
- only boot enable in your dtb the cores with the same type than the boot core
- check in your device tree which devices are assign to the ppi_cluster not available and disable them
This kind of configuration will be tricky to handle so it will depend on which devices have interrupts that can only be fired on one cluster type.
Sorry but I cannot right now check in the device tree and do those checks myself but if you do get the idea send back a mail :-)
Cheers
Bertrand
> };
>
> I've attached the device tree and include files just in case you'd like to take a look at those.
>
> Thanks so much for your help! I've been at this for weeks with very little progress.
>
> Cheers,
> Brad
>
>
> On Tue, Jul 12, 2022 at 10:05 AM Bertrand Marquis <Bertrand.Marquis@xxxxxxx> wrote:
> Hi Brad,
>
> > On 11 Jul 2022, at 19:38, Brad Churchwell <brad@xxxxxxxxxxxx> wrote:
> >
> > Hello,
> >
> > I've been trying to get Xen to boot dom0 with my kernel for weeks on an rk3399 based board and thought I'd reach out for help. It looks like either Xen is not properly recreating the device tree or the interrupt controller is just failing. The hypervisor boots but falls to initramfs because it cannot find the root device (nvme on pcie). Any help would be greatly appreciated. Here is the complete boot log
>
> From the logs you have an issue with the interrupt controller and the fact that your NVME is behind PCIE and as such depends on ITS is probably not helping.
>
> I would suggest to try to boot on usb as root fs for a try.
>
> Also it could be useful to compare the device tree on xen and without xen to understand what is going on (using /proc/device-tree).
>
> Xen seems to be ok but Linux is not happy with interrupts and is showing several issues around this area on your logs.
> Could you show us an extract of your device tree around the gic declaration ?
>
> Cheers
> Bertrand
>
>
> IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
> <rk3399-zaku.dts><rk3399-zaku.dtsi><rk3399.dtsi>
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
|
