Re: [Xen-devel] [PATCH] xen/arm64: Use __flush_dcache_area instead of __flush_dcache_all

[Mark Rutland <mark.rutland@xxxxxxx>]

On Tue, Oct 14, 2014 at 10:35:23AM +0100, Ian Campbell wrote:
> On Tue, 2014-10-14 at 10:21 +0100, Mark Rutland wrote:
> > Hi Roy,
> > 
> > [...]
> > 
> > > It seems that for Xen we do need to flush the FDT as well - I get a
> > > variety of crashes
> > > with a corrupt FDT when cache state is modeled on the FVP model, and
> > > Suravee sees similar
> > > behavior on Seattle. I was not expecting this, as I looked at the code
> > > in Xen and the caches/TLB
> > > are enabled quite early on, before the FDT is accessed by Xen.  I then
> > > looked at the mappings
> > > used by  edk2 and Xen, and found some differences.  Even after
> > > modifying edk2 to use the same
> > > configuration as Xen, the flushing of the FDT is still required. Xen
> > > and edk2 use the same memory
> > > attributes  in the MAIR_EL2 register (0xFF), but had different
> > > sharing, access perm, and nG settings.
> > 
> > I don't think the access perm or nG settings should have any effect, but
> > the shareability forms part of the memory attributes (along with the
> > memory type and cacheability), and there are several rules that apply
> > when accessing a memory location with mismatched attributes. See the
> > ARMv8 ARM - The AArch64 Application Level Memory Model - Mismatched
> > memory attributes.
> > 
> > In Linux we're likely getting lucky, and the shareability we use varies
> > for an SMP or UP kernel. So we need maintenance in at least one of those
> > cases. This would also apply to any initrd or other image.
> > 
> > Do you happen to know the shareability used by EDK2 and Xen?
> 
> Xen maps everything inner-shareable. Dunno about EDK2.

Ok. That matches what an SMP Linux kernel will do, so it looks like
we're just getting lucky with Linux. I'lll have a play and see if I can
trigger similar issues.

> Is the real issue here not a lack of specification for some corner cases
> of the boot protocol? Can we get that fixed somehow?

To an extent, yes. We can try to fix up the Linux side with patche to
Documentation/arm64/booting.txt. As far as I am aware, for UEFI that
will require membership of the UEFI forum.

> Part of me wants to suggest that UEFI (and bootloaders generally) ought
> to be cleaning caches for anything they have loaded into RAM before
> launching an OS as a matter of good hygiene.

In general, yes.

Unfortunately, UEFI can't perform the maintenance in this case, because
the stub modifies things. I was under the impression it copied and
modified the FDT to embed the command line -- UEFI has no visibiltiy of
this and therefore cannot be in charge of flushing it. So in this case,
the stub needs to be thought of as the bootloader, and needs to be in
charge of any required maintenance.

There are a tonne of subtleties here, and certain properties we would
like (e.g. a completely clean cache hierarchy upon entry to the OS)
aren't necessarily possible to provide in general (thanks to the wonders
of non-architected system level caches, interaction with bootloaders,
etc).

Mark.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel