Re: [Xen-devel] REGRESSION: Xen 4.13 RC5 fails to bootstrap Dom0 on ARM

[Roman Shaposhnik <roman@xxxxxxxxxx>]

Hi,

On Wed, Dec 18, 2019 at 4:56 AM Julien Grall <julien@xxxxxxx> wrote:
> > So that is, in fact, my first question -- why is Xen not showing
> > available memory in xl info?
>
> I am not entirely sure what exact information you want.
>
> The output you dumped above contain the available memory for the memory
> (see "free_memory").
>
> Are you looking from something different?

Just to be clear: I was giving 2G via devicetrees (the same device
trees that would
make Linux detect 2G of RAM) hence I was expecting xl info to show that. Instead
I only got 1120M shown by xl info.

> On 18/12/2019 00:04, Roman Shaposhnik wrote:
> >          memory {
> >                  device_type = "memory";
> >                  reg = <0x0 0x0 0x0 0x5e00000 0x0 0x5f00000 0x0 0x1000
> > 0x0 0x5f02000 0x0 0xefd000 0x0 0x6e00000 0x0 0x60f000 0x0 0x7410000
> > 0x0 0x1aaf0000 0x0 0x21f00000 0x0 0x100000 0x0 0x22000000 0x0
> > 0x1c000000>;
> >          };
> >
> >          reserved-memory {
> >                  ranges;
> >                  #size-cells = <0x2>;
> >                  #address-cells = <0x2>;
> >
> >                  ramoops@21f00000 {
> >                          ftrace-size = <0x20000>;
> >                          console-size = <0x20000>;
> >                          reg = <0x0 0x21f00000 0x0 0x100000>;
> >                          record-size = <0x20000>;
> >                          compatible = "ramoops";
> >                  };
> >
> >                  linux,cma {
> >                          linux,cma-default;
> >                          reusable;
> >                          size = <0x0 0x8000000>;
> >                          compatible = "shared-dma-pool";
> >                  };
> >          };
> >
> > If you look at the REG -- it does now add up to 2Gb, but booting Xen
> > with it has exactly the
> > same effect as booting it with: reg = <0x0 0x0 0x0 0x80000000>;\
>
> If you boot Xen using EFI, the memory information wil come from EFI and
> the DT node will be ignored. So unless UEFI is able to pick up the
> modification of the DT memory node, modifying the DT is not going to
> affect anything.

That's a good point, but given that I always go through GRUB, I was
expecting devicetree command to completely overshadow whatever
information UEFI may have. Am I wrong?

> > I am attaching a full log, and I see the following in the logs:
> >
> > (XEN) Allocating 1:1 mappings totalling 720MB for dom0:
> > (XEN) BANK[0] 0x00000008000000-0x0000001c000000 (320MB)
> > (XEN) BANK[1] 0x00000040000000-0x00000058000000 (384MB)
> > (XEN) BANK[2] 0x0000007b000000-0x0000007c000000 (16MB)
> >
> > Which sort of makes sense, I guess -- but I still don't understand
> > where all these ranges
> > are coming from and how come Xen doesn't see the full 2Gb even with various
> > devicetrees I tried.
>
> The range aboves describe the memory range given to Dom0. For all the
> memory given to Xen,m you want to look at the top of your log:
>
> (XEN) Checking for initrd in /chosen
> (XEN) RAM: 0000000000000000 - 0000000005dfffff
> (XEN) RAM: 0000000005f00000 - 0000000006dfefff
> (XEN) RAM: 0000000006e00000 - 000000000740efff
> (XEN) RAM: 0000000007410000 - 000000001db8dfff
> (XEN) RAM: 00000000350f0000 - 000000003dbd2fff
> (XEN) RAM: 000000003dbd3000 - 000000003dffffff
> (XEN) RAM: 0000000040000000 - 000000005a653fff
> (XEN) RAM: 000000007ada0000 - 000000007ada3fff
> (XEN) RAM: 000000007aea8000 - 000000007afa9fff
> (XEN) RAM: 000000007afaa000 - 000000007ec73fff
> (XEN) RAM: 000000007ec74000 - 000000007fdddfff
> (XEN) RAM: 000000007fdde000 - 000000007fea5fff
> (XEN) RAM: 000000007fea6000 - 000000007ff6dfff
> (XEN) RAM: 000000007ffff000 - 000000007fffffff
>
> Looking at the differences with the Linux logs, there is indeed some
> memory not detected by Xen.
>
> On Xen, we only consider usuable memory any EFI description with
> EfiConventionalMemory, EfiBootServicesCode and EfiBootServicesData.
>
> Linux include more type here, so this may explain why we see a difference.
>
> While Looking at it, I have also noticed that we don't seem to care
> about the memory attribute. I suspect this could be another latent issue
> in Xen if the attribute does not match.

Anything I can do to help debug this? I can run any kind of debug builds, etc.
if needed.

I mean -- at this point it would be really great to get HiKey back to the status
of Xen-on-ARM developer board.

> > Any ideas here would be greatly apprecaited!
> >
> > Thanks,
> > Roman.
> >
> > P.S. Any guess at what these mean?
> >
> > (XEN) traps.c:1973:d0v0 HSR=0x93880006 pc=0x00ffff87355558
> > gva=0xffff872f2000 gpa=0x000000000f0000
> > (XEN) traps.c:1973:d0v0 HSR=0x93880006 pc=0x00ffffb734e558
> > gva=0xffffb72eb000 gpa=0x000000000f0000
> > (XEN) traps.c:1973:d0v0 HSR=0x93880006 pc=0x00ffff8f9d2558
> > gva=0xffff8f96f000 gpa=0x000000000f0000
>
> It means that Linux has tried to access something that has not been
> mapped in stage-2. As Dom0 is mapped 1:1, the GPA also give you the host
> physical address. In this case, it is trying to access 0xf0000.
>
> This seems to belong to the RAM, but this part has not been allocated to
> Dom0.

Got it! Thank you! Am I correct in guessing that this can only come from
a driver of some sorts trying to tickle the hardware? IOW, I should be
looking for some abnormalities in my linux kernel messages to try and
see what this could be.

> You may get more information from Dom0 if you add earlycon=xenboot on
> your linux command line. This will give you some output using the
> earlyconsole before the console subsytem is actually initialize.

Will do!

Thanks,
Roman.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel