RE: Aligning Xen to physical memory maps on embedded systems

[Levenglick Dov <Dov.Levenglick@xxxxxxxxxxxxxxxx>]

> 
> (+ Penny, Wei and Luca)
> 
> > On 23 Feb 2021, at 01:52, Stefano Stabellini <sstabellini@xxxxxxxxxx> wrote:
> >
> > On Mon, 22 Feb 2021, Levenglick Dov wrote:
> >>>> The system has 2GB of RAM (0x00000000 - 0x80000000) of which Xen
> >>>> and the DomU have an allocation of 1.25GB, per this memory map:
> >>>> 1. DomU1: 0x60000000 - 0x80000000
> >>>> 2. DomU2: 0x40000000 - 0x60000000
> >>>> 3. Xen: 0x30000000 - 0x40000000
> >>>
> >>> How did you tell Xen which regions is assigned to which guests? Are
> >>> your domain mapped 1:1 (i.e guest physical address == host physical
> address)?
> >>
> >> I am working on a solution where if the "xen,domain" memory has
> >> #size-cell cells the content is backward compatible. But if it
> >> contains (#address-cells + #size-cells), the address cells should be
> considered the physical start address.
> >> During the mapping of the entire address space insetup_mm(), the
> >> carved out addresses would be added to the  reserved memory address
> >> space. When the DomU is to be created, this physical space would be
> >> mapped to it. The virtual addresses are less of an issue and needn't be
> mapped 1x1 (although they could be).
> >
> > As of today neither upstream Xen nor the Xilinx Xen tree come with the
> > feature of allowing the specification of an address range for dom0less
> > guests.
> >
> > The only thing that Xilinx Xen allows, which is not upstream yet, is
> > the ability of creating dom0less guests 1:1 mapped using the "direct-map"
> > property. But the memory allocation is still done by Xen (you can't
> > select the addresses).
> >
> > Some time ago I worked on a hacky prototype to allow the specification
> > of address ranges, see:
> >
> > http://xenbits.xenproject.org/git-http/people/sstabellini/xen-unstable
> > .git direct-map-2 from 7372466b21c3b6c96bb7a52754e432bac883a1e3
> onward.
> >
> > In particular, have a look at "xen/arm: introduce 1:1 mapping for
> > domUs". The work is not complete: it might not work depending on the
> > memory ranges you select for your domUs. In particular, you can't
> > select top-of-RAM addresses for your domUs. However, it might help you
> > getting started.
> >
> >
> >>>> I am able to support True Dom0-less by means of the patch/hack
> >>>> demonstrated By Stefano Stabellini at
> >>> https://youtu.be/UfiP9eAV0WA?t=1746.
> >>>>
> >>>> I was able to forcefully put the Xen binary at the address range
> >>>> immediately below 0x40000000 by means of modifying
> get_xen_paddr()
> >>>> -
> >>> in itself an ugly hack.
> >>>>
> >>>> My questions are:
> >>>> 1. Since Xen performs runtime allocations from its heap, it is allocating
> >>>>    downwards from 0x80000000 - thereby "stealing" memory from
> DomU1.
> >>>
> >>> In theory, any memory reserved for domains should have been carved
> >>> out from the heap allocator. This would be sufficient to prevent Xen
> >>> allocating memory from the ranges you described above.
> >>>
> >>> Therefore, to me this looks like a bug in the tree you are using.
> >>
> >> This would be a better approach, but because Xen perform allocations
> >> from its heap prior to allocating memory to DomU - and since it
> >> allocates from the top of the heap - it is basically taking memory that I
> wanted to set aside for the DomU.
> >
> > Yeah, this is the main problem that my prototype above couldn't solve.

Stephano: Is the approach that I previously described a feasible one?
  1. Mark the addresses that I want to set aside as reserved
  2. When reaching the proper DomU, map them and then use the mapping
This approach would solve the heap issue
> >
> 
> Wei and Penny are working on direct map and static allocation to fit
> embedded use cases an might have more answer there.

Bertrand, Wei and Penny: Is there a "sneak preview"? I'd be happy to start 
backporting to Xen 4.11

> 
> On the fix from Stefano explained in the video, Luca Fancellu made a patch to
> propose a long term solution and will push it upstream next week.

Bertrand: Do You know which commit ID this is? Since I'm working on a Xilinx 
fork, I am out of touch with the goings of the main tree.


Thanks,
Dov
The information in this e-mail transmission contains proprietary and business 
sensitive information.  Unauthorized interception of this e-mail may constitute 
a violation of law. If you are not the intended recipient, you are hereby 
notified that any review, dissemination, distribution or duplication of this 
communication is strictly prohibited. You are also asked to contact the sender 
by reply email and immediately destroy all copies of the original message.