Re: Hand over of the Xen shared info page

[Anastasiia Lukianenko <anastasiia_lukianenko@xxxxxxxx>]

Hi Julien!

On Thu, 2021-05-13 at 09:37 +0100, Julien Grall wrote:
> 
> On 13/05/2021 09:03, Anastasiia Lukianenko wrote:
> > Hi all,
> 
> Hi,
> 
> > The problem described below concerns cases when a shared info page
> > needs to be handed over from one entity in the system to another,
> > for
> > example, there is a bootloader or any other code that may run
> > before
> > the guest OS' kernel.
> > Normally, to map the shared info page guests allocate a memory page
> > from their RAM and map the shared info on top of it. Specifically
> > we
> > use XENMAPSPACE_shared_info memory space in  XENMEM_add_to_physmap
> > hypercall.  As the info page exists throughout the guest existence
> > this
> > doesn't hurt the guest, but when the page gets out of accounting,
> > e.g.
> > after bootloader jumps to Linux and the page is not handed over to
> > it,
> > the mapped page becomes a problem.
> > Consider the case with U-boot bootloader which already has Xen
> > support.
> > The U-boot’s Xen guest implementation allocates a shared info page
> > between Xen and the guest domain and U-boot uses domain's RAM
> > address
> > space to create and map the shared info page by using
> > XENMEM_add_to_physmap hypercall [1].
> > 
> > After U-boot transfers control to the operating system (Linux,
> > Android,
> > etc), the shared info page is still mapped in domain’s address
> > space,
> > e.g. its RAM. So, after we leave U-boot, this page becomes just an
> > ordinary memory page from Linux POV while it is still a shared info
> > page from Xen POV. This can lead to undefined behavior, errors etc
> > as
> > Xen can write something to the shared info page and when Linux
> > tries to
> > use it - data corruption may happen.
> > This happens because there is no unmap function in Xen API to
> > remove an
> > existing shared info page mapping. We could use only hypercall
> > XENMEM_remove_from_physmap which eventually will create a hole in
> > the
> > domain's RAM address space which may also lead to guest’s crash
> > while
> > accessing that memory.
> 
> The hypercall XENMEM_remove_from_physmap is the correct hypercall
> here 
> and work as intented. It is not Xen business to keep track what was
> the 
> original page (it may have been RAM, device...).
> 
> The problem here is the hypercall XENMEM_add_to_physmap is misused
> in 
> U-boot. When you give an address for the mapping you are telling Xen 
> "Here a free region to map the share paged". IOW, Xen will throw
> away 
> whatever was before because that was you asked.
> 
> If you want to map in place of the RAM page, then the correct
> approach 
> is to:
>    1) Request Xen to remove the RAM page from the P2M
>    2) Map the shared page
>    /* Use it */
>    3) Unmap the shared page
>    4) Allocate the memory
> 
> You can avoid 1) and 4) by finding free region in the address space.
> 
> > 
> > We noticed this problem and the workaround was implemented using
> > the
> > special GUEST_MAGIC memory region [2].
> > 
> > Now we want to make a proper solution based on GUEST_MAGIC_BASE,
> > which
> > does not belong to the guest’s RAM address space [3]. Using the
> > example
> > of how offsets for the console and xenstore are implemented, we can
> > add
> > a new shared_info offset and increase the number of magic pages [4]
> > and
> > implement related functionality, so there is a similar API to query
> > that magic page location as it is done for console PFN and others.
> 
> They are not the same type. The console PFN points memory already 
> populated in the guest address space.
> 
> For the domain shared page, this is memory belonging to Xen that you 
> will map in your address space. A domain can map it anywhere it
> wants.
> 
> > This approach would allow the use of the XENMEM_remove_from_physmap
> > hypercall without creating gaps in the RAM address space for Xen
> > guest
> > OS [5].
> 
> See above to prevent the gap. I appreciate this means a superpage
> may 
> get shattered.
> 
> The alternative is for U-boot to go through the DT and infer which 
> regions are free (IOW any region not described).

Thank you for interest in the problem and advice on how to solve it.
Could you please clarify how we could find free regions using DT in U-
boot?

Regards,
Anastasiia
> 
> Cheers,
>