RE: Xen Memory Sharing Query

[Marc Bonnici <Marc.Bonnici@xxxxxxx>]

Hi Andrew,

Thank you very much for your detailed reply, that make things 
a lot clearer. I did have a few follow up questions.

> gnttab v2 is horribly more complicated.  

With v2, do the high level behaviours change much from 
what you have outlined here? Do you expect in the long term 
for v2 to be the preferred implementation or are they more like 
alternatives?


> While a gref is mapped, domA is not permitted to edit the 
> associated entry.  

So IIUC once the gref is mapped then domA can't make any changes 
to the entry at all, (or at least they won't be reflected). So if 
domA wants to make any modifications (extend the shared memory 
region, change permissions etc.), then it would just create another
entry and share the new gref? 


> From a grant perspective, Xen doesn't enforce any policy.  domA's grefs
> can be mapped in the manner permitted by what domA wrote into the grant
> table.

So does this mean that if domA grants access to domB for a given frame, 
and then added a new entry in its grant table with the same frame details 
but with "domid = domC" instead. This would be allowed? And if so, would this 
result in a 3-way shared buffer?

And finally a similar scenario, if a frame was shared from domA to domB, 
would domB be able to add an entry in its own grant table to share the 
same frame with domC and end up with the same outcome?

Thanks

Kind Regards
Marc Bonnici

> -----Original Message-----
> From: Andrew Cooper <Andrew.Cooper3@xxxxxxxxxx>
> Sent: 15 July 2022 18:29
> To: Marc Bonnici <Marc.Bonnici@xxxxxxx>; xen-
> devel@xxxxxxxxxxxxxxxxxxxx
> Subject: Re: Xen Memory Sharing Query
> 
> On 15/07/2022 16:56, Marc Bonnici wrote:
> > Hi All,
> >
> > I was wondering if someone could help me understand some of the
> rules of the
> > memory sharing implementation in Xen?
> >
> > Specifically I'm looking to understand what restrictions Xen
> places on
> > granting access from one Dom to another from Xen's perspective,
> and what types
> > of grant requests would be allowed/rejected by Xen?
> >
> > I.e. How would the situation be handled if the same frame of
> memory was attempted
> > to be shared multiple times?
> >
> > As an example scenario, DomA shares 1 physical page of memory in a
> transaction
> > with DomB. And then without releasing any memory, DomA attempts to
> share
> > another region of memory, which includes the same physical page of
> the previous share
> > with DomB again. This would result in two concurrent shares
> containing an overlap.
> >
> > Apologies if I've missed something but is there any documentation
> / threat model
> > that would cover these types of scenarios? So far I have been
> trying to read through
> > the code but was wondering if there is something else I could
> refer to help me
> > with my understanding?
> 
> There's nothing adequately written down.  It ought to live in sphinx
> docs, but my copious free time is non-existent for speculative
> security
> reasons.
> 
> This all pertains to gnttab v1 which is the only supported one on
> ARM
> right now.  gnttab v2 is horribly more complicated.  Refer to
> https://github.com/xen-
> project/xen/blob/master/xen/include/public/grant_table.h#L132-L186
> 
> When DomA and DomB are set up and running, they each have a grant
> table.  The grant table is some shared memory (of Xen's) mapped into
> the
> guest, and is a bidirectional communication interface between the
> guest
> kernel and Xen.
> 
> The guest kernel logically owns the grant table, and it's a simple
> array
> of grant entries.  Entries 0 thru 7 are reserved for system use, and
> indeed two entries (one for xenstore, one for xenconsole) are set up
> on
> the guest kernel's behalf by the domain builder.  Entries 8 thru $N
> are
> entirely under the guest's control.
> 
> A guest kernel (domA) creates a grant by filling in a grant table
> entry,
> and passing the grant reference (the entry's index in the table) to
> some
> other entity in the system (in this case, domB).
> 
> The grant table entry is formed of:
> 
> u16 flags
> u16 domid
> u32 frame
> 
> so for domA to grant a frame to domB, it would pick a free gref (any
> entry in the table with flags=0) and fill in:
> 
> frame = f
> domid = domB
> smp_wmb()
> flags = GTF_permit_access (think "grant usable")
> 
> GFT_readonly is another relevant flag that domA might choose to set.
> 
> Then, domB would take the gref it has been given by domA, and make a
> gnttab_op_map() hypercall, passing {domA, gref} as an input.
> 
> Xen looks up gref in domA's grant table, checks e.g. domA granted
> access
> to domB, and if everything is happy, sets the the
> GFT_{reading,writing}
> flags (as appropriate) in flags.  This tells domA that the grant is
> currently mapped readably and/or writeably.
> 
> Later, when domB unmaps the grant, Xen clears the
> GFT_{reading,writing}
> bits, telling domA that the grant is no longer in use.
> 
> DomA then clears GTF_permit_access to mark this as gref as invalid,
> and
> can then free the frame.
> 
> 
> Now, that's the simple overview.  To answer some of your specific
> questions:
> 
> DomA is perfectly free to grant away the same frame multiple times.
> DomA does this by writing multiple different grefs with the same
> frame
> field.  These grefs could be to the same, or different domains, and
> can
> have any (valid) combination of flags.
> 
> DomB is perfectly free to map the same gref multiple times.  This is
> actually a necessity for x86 PV guests, because of how we reference
> count pagetable entries.  It is not necessary for any kind of guest
> of
> HVM guest (x86 or ARM) because of how guest physical address space
> works.
> 
> IMO it should have been restricted when the HVM ABI was designed,
> but
> alas.  In practice, Xen has an internal refcount which prevents a
> gref
> being mapped more than 127 times IIRC.
> 
> While a gref is mapped, domA is not permitted to edit the associated
> entry.  Doing so shouldn't cause a security violation (Xen has a
> local
> copy of the entry in the maptrack table), but will at least confuse
> diagnostics of the granted state.
> 
> Importantly, and what may come as a surprise, is that domA has no
> way to
> revoke a currently-mapped grant.  Fixing this limitation has been
> discussed several times; there are some very complicated corner
> cases,
> and I'm not aware of any work having started in earnest.
> 
> Xen does have logic to unmap grants of VMs which have shut down (for
> whatever reason) with grants still mapped.  This prevents deadlocks
> (e.g. two domains grant to each other, then both crash deliberately).
> 
> 
> From a grant perspective, Xen doesn't enforce any policy.  domA's
> grefs
> can be mapped in the manner permitted by what domA wrote into the
> grant
> table.
> 
> If you want to get into policies that Xen may enforce, that would be
> a
> discussion about XSM, Xen Security Modules.
> 
> Does any of this help?
> 
> ~Andrew