Re: [Xen-devel] xen/arm: Alternative start of day cache coherency

[Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>]

On Thu, 23 Jan 2014, Ian Campbell wrote:
> On Thu, 2014-01-23 at 13:55 +0000, Stefano Stabellini wrote:
> > On Thu, 23 Jan 2014, Ian Campbell wrote:
> > > On Wed, 2014-01-22 at 17:32 +0000, Stefano Stabellini wrote:
> > > > > +int xc_domain_cacheflush(xc_interface *xch, uint32_t domid)
> > > > > +{
> > > > > +    DECLARE_DOMCTL;
> > > > > +    domctl.cmd = XEN_DOMCTL_cacheflush;
> > > > > +    domctl.domain = (domid_t)domid;
> > > > > +    domctl.u.cacheflush.start_mfn = 0;
> > > > > +    return do_domctl(xch, &domctl);
> > > > > +}
> > > > 
> > > > Do we really need to flush the entire p2m, or just things we have
> > > > written to?
> > > 
> > > I think we need to flush everything (well, all RAM backed pages, the
> > > patch skips everything else).
> > > 
> > > Even things which we haven't explicitly written to will have been
> > > scrubbed and therefore have scrubbed data in the cache but data
> > > belonging to the previous owner in actual RAM. So we would really want
> > > to clean in that case too.
> > > 
> > > We could do the clean at scrub time which arguably be better anyway and
> > > would potentially allows us to only invalidate instead of clean
> > > +invalidate some subset of pages, but we would need to track which sort
> > > of page was which -- e.g. with a special p2m type for a page which had
> > > been foreign mapped or some other bit of metadata.
> > 
> > That seems like the way to go.
> 
> I'm not convinced actually, and in any case, not for 4.4...
> 
> > > > > diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
> > > > > index 85ca330..f35ed57 100644
> > > > > --- a/xen/arch/arm/p2m.c
> > > > > +++ b/xen/arch/arm/p2m.c
> > > > > @@ -228,15 +228,26 @@ enum p2m_operation {
> > > > >      ALLOCATE,
> > > > >      REMOVE,
> > > > >      RELINQUISH,
> > > > > +    CACHEFLUSH,
> > > > >  };
> > > > >  
> > > > > +static void do_one_cacheflush(paddr_t mfn)
> > > > > +{
> > > > > +    void *v = map_domain_page(mfn);
> > > > > +
> > > > > +    flush_xen_dcache_va_range(v, PAGE_SIZE);
> > > > > +
> > > > > +    unmap_domain_page(v);
> > > > > +}
> > > > 
> > > > A pity that we need to map a page just to flush the dcache.  It could be
> > > > expensive, especially if we really have to map every single guest mfn.
> > > 
> > > Remember that this is basically free for arm64 and for arm32 we actually
> > > map 2MB regions and cache, so it is only actually one map per 2MB
> > > region.
> > 
> > Even with 2MB at a time it is easy for this to become really slow. A 4GB
> > guest would need 2048 iterations of map/flush/unmap. I don't have any
> > numbers but I bet they won't look good.
> 
> It happens once during domain build and the delay it isn't noticeable to
> me compared with the rest of the build process.
> 
> > At least if it was combined with the RAM scrub we would save the 2048
> > map/unmap.
> 
> The scrub has to map it in exactly the same way.

Right, since we are already doing it once, why do it twice? :)

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel