Re: [Xen-devel] [RFC] DOMCTL_memattrs_op : a new DOMCTL to play with stage-2 page attributes

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Mon, 3 Jul 2017, Zhongze Liu wrote:
> Hi Julien,
> 
> 2017-07-03 19:16 GMT+08:00 Julien Grall <julien.grall@xxxxxxx>:
> > Hi,
> >
> > On 01/07/17 10:16, Zhongze Liu wrote:
> >>>
> >>> On the ARM side, we are missing BUFFERABLE and WRITEALLOC. I don't know
> >>> how they map to these tags, which comes from the x86 world. Maybe we
> >>> should just add them separately as ARM only, like:
> >>>
> >>>   /* bufferable, ARM only */
> >>>   #define XEN_DOMCTL_MEMATTRS_BUFFERABLE 0x08U
> >>>   /* write alloc, ARM only */
> >>>   #define XEN_DOMCTL_MEMATTRS_CACHE_WA   0x09U
> >>>
> >>> Theoretically, we could say XEN_DOMCTL_MEMATTRS_UC means "BUFFERABLE" on
> >>> ARM and XEN_DOMCTL_MEMATTRS_SUC means "UNCACHED", because that's
> >>> actually what they correspond to I think. However using x86 names for
> >>> ARM caching attributes is very confusing and error prone. So I would
> >>> prefer introducing separate tags for ARM and x86. However, reusing
> >>> XEN_DOMCTL_MEMATTRS_UC, XEN_DOMCTL_MEMATTRS_CACHE_WT and
> >>> XEN_DOMCTL_MEMATTRS_CACHE_WB as Zhongze did in this proposal would be OK
> >>> for me.
> >
> >
> > When I read bufferable it is unclear if you speak about normal memory or
> > device. I am looking at renaming the memory attribute with prefixing them
> > with the type memory.
> >
> > For instance BUFFERABLE would be renamed to NORMAL_NC...
> >
> >>>
> >>> Julien, what do you think?
> >
> >
> > I will only speak about ARM as my knowledge is very limited on x86.
> >
> > For ARM, the resulting memory attribute is a combination of stage-1 and
> > stage-2 (see Table D4-43 in ARM DDI 0487B.a). It adds further restriction to
> > the memory attributes defined by the Guest in its page-tables.
> >
> > This means that even the memory attribute used in stage-2 is normal
> > cacheable, a guest is free to make it non-cacheable via stage-1 page table.
> > This is not really clear in the description of the DOMCTL what is the real
> > purpose. Is it restricting possibility of the guest?
> 
> Yes. this only deals with the stage-2 table entries, and thus only serves as a
> restriction on what the DomU's can do. And the DomU's can do whatever they
> want to their stage-1 table entries, as long as they don't try to
> break the restrictions.
> 
> >
> > Now, looking at the description, this domctl will be called after we mapped
> > the RAM in the guest memory. So you will switch from write-back cacheable to
> > another memory attribute. I think this will require cache maintainance to
> > remove potential stall cache line.
> >
> > Furthermore, you don't have any restriction on when this domctl will be
> > called. It would be possible to call it when the guest is running or called
> > on a range with memory attribute already changed. This will require some
> > thoughts on how to do the cache maintenance.
> >
> > Finally, Xen ARM64 will always have the whole RAM memory mapped in Xen with
> > write-allocate memory attribute. This may result a memory attribute mismatch
> > if the region is accessed by Xen (see B2.8).
> 
> Actually, I was considering whether the shared areas should be set up during
> domain construction, I think this will make better sense.
> 
> @Stebellini: what do you think?

Yes, it makes sense to set them up during domain construction.

However what Julien was also pointing out is that if the user requests
memory attributes different from the one used by Xen (because Xen will
also have the same pages already mapped in its own pagetables
regardless), then it will be an issue for Xen to access them: it could
result in a memory attribute mismatch.

We would need to make sure somehow that those pages are not accessed by
Xen for as long as they are shared between guests with different memory
attributes in the guest stage2 pagetables.

This is another reason for not implementing different memory attributes
right now :-)

I would add a note about this unsolved problem in future versions of
this document.


> >
> > This may take sometimes to get the implementation of the DOMCTL right. So I
> > would rather focus to be able to share page between guest and an
> > future-proof toolstack interface.
> >
> > If you still have time at the end of the GSOC, you can look at using
> > different memory attributes
> 
> Agree. That's what Stabellini has also suggested me to do. From now on, I'll 
> be
> focusing more on the other parts of this project, while waiting for
> more feedback
> on how to do this attribute stuff right.

Yes, good idea, thank you. This is delicate and will likely take some
time, without actually affecting the rest of the code much.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel