Re: Hand over of the Xen shared info page

[Julien Grall <julien@xxxxxxx>]

On 13/05/2021 09:03, Anastasiia Lukianenko wrote:
> Hi all,
Hi,

> The problem described below concerns cases when a shared info page
> needs to be handed over from one entity in the system to another, for
> example, there is a bootloader or any other code that may run before
> the guest OS' kernel.
> Normally, to map the shared info page guests allocate a memory page
> from their RAM and map the shared info on top of it. Specifically we
> use XENMAPSPACE_shared_info memory space in  XENMEM_add_to_physmap
> hypercall.  As the info page exists throughout the guest existence this
> doesn't hurt the guest, but when the page gets out of accounting, e.g.
> after bootloader jumps to Linux and the page is not handed over to it,
> the mapped page becomes a problem.
> Consider the case with U-boot bootloader which already has Xen support.
> The U-boot’s Xen guest implementation allocates a shared info page
> between Xen and the guest domain and U-boot uses domain's RAM address
> space to create and map the shared info page by using
> XENMEM_add_to_physmap hypercall [1].
>
> After U-boot transfers control to the operating system (Linux, Android,
> etc), the shared info page is still mapped in domain’s address space,
> e.g. its RAM. So, after we leave U-boot, this page becomes just an
> ordinary memory page from Linux POV while it is still a shared info
> page from Xen POV. This can lead to undefined behavior, errors etc as
> Xen can write something to the shared info page and when Linux tries to
> use it - data corruption may happen.
> This happens because there is no unmap function in Xen API to remove an
> existing shared info page mapping. We could use only hypercall
> XENMEM_remove_from_physmap which eventually will create a hole in the
> domain's RAM address space which may also lead to guest’s crash while
> accessing that memory.
The hypercall XENMEM_remove_from_physmap is the correct hypercall here 
and work as intented. It is not Xen business to keep track what was the 
original page (it may have been RAM, device...).
The problem here is the hypercall XENMEM_add_to_physmap is misused in 
U-boot. When you give an address for the mapping you are telling Xen 
"Here a free region to map the share paged". IOW, Xen will throw away 
whatever was before because that was you asked.
If you want to map in place of the RAM page, then the correct approach 
is to:
  1) Request Xen to remove the RAM page from the P2M
  2) Map the shared page
  /* Use it */
  3) Unmap the shared page
  4) Allocate the memory

You can avoid 1) and 4) by finding free region in the address space.

> We noticed this problem and the workaround was implemented using the
> special GUEST_MAGIC memory region [2].
>
> Now we want to make a proper solution based on GUEST_MAGIC_BASE, which
> does not belong to the guest’s RAM address space [3]. Using the example
> of how offsets for the console and xenstore are implemented, we can add
> a new shared_info offset and increase the number of magic pages [4] and
> implement related functionality, so there is a similar API to query
> that magic page location as it is done for console PFN and others.
They are not the same type. The console PFN points memory already 
populated in the guest address space.
For the domain shared page, this is memory belonging to Xen that you 
will map in your address space. A domain can map it anywhere it wants.
> This approach would allow the use of the XENMEM_remove_from_physmap
> hypercall without creating gaps in the RAM address space for Xen guest
> OS [5].
See above to prevent the gap. I appreciate this means a superpage may 
get shattered.
The alternative is for U-boot to go through the DT and infer which 
regions are free (IOW any region not described).
Cheers,

--
Julien Grall