Re: E820 memory allocation issue on Threadripper platforms

[Marek Marczykowski-Górecki <marmarek@xxxxxxxxxxxxxxxxxxxxxx>]

On Fri, Jan 19, 2024 at 02:50:38PM +0100, Jan Beulich wrote:
> On 19.01.2024 14:40, Marek Marczykowski-Górecki wrote:
> > On Thu, Jan 18, 2024 at 01:23:56AM -0500, Patrick Plenefisch wrote:
> >> On Wed, Jan 17, 2024 at 3:46 AM Jan Beulich <jbeulich@xxxxxxxx> wrote:
> >>> On 17.01.2024 07:12, Patrick Plenefisch wrote:
> >>>> As someone who hasn't built a kernel in over a decade, should I figure
> >>> out
> >>>> how to do a kernel build with CONFIG_PHYSICAL_START=0x2000000 and report
> >>>> back?
> >>>
> >>> That was largely a suggestion to perhaps allow you to gain some
> >>> workable setup. It would be of interest to us largely for completeness.
> >>>
> >>
> >> Typo aside, setting the boot to 2MiB works! It works better for PV
> > 
> > Are there any downsides of running kernel with
> > CONFIG_PHYSICAL_START=0x200000? I can confirm it fixes the issue on
> > another affected system, and if there aren't any practical downsides,
> > I'm tempted to change it the default kernel in Qubes OS.
> 
> There must have been a reason to make the default 16Mb. You may want
> to fish out the commit doing so ... 

https://git.kernel.org/torvalds/c/ceefccc93932b920

    Default CONFIG_PHYSICAL_START and CONFIG_PHYSICAL_ALIGN each to 16 MB,
    so that both non-relocatable and relocatable kernels are loaded at
    16 MB by a non-relocating bootloader.  This is somewhat hacky, but it
    appears to be the only way to do this that does not break some some
    set of existing bootloaders.

    We want to avoid the bottom 16 MB because of large page breakup,
    memory holes, and ZONE_DMA.  Embedded systems may need to reduce this,
    or update their bootloaders to be aware of the new min_alignment field.

Large pages (in practice) do not apply to PV dom0, but other points
could in theory. That said, I checked few other systems and I don't see
any reserved regions there (there is large usable region at 0x100000,
other reserved regions are near the 4GB boundary).
This isn't very representative sample, though...

> In Qubes, though, I understand
> you're always running with Xen underneath, so unless this same kernel
> is also needed to run in HVM guests, some of whatever the reasons may
> have been may go away.

The same kernel is used for PVH/HVM guests too.

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

Attachment: signature.asc
Description: PGP signature