Re: [Xen-devel] Problems with merlot* AMD Opteron 6376 systems (Was Re: stable trees (was: [xen-4.2-testing test] 58584: regressions))

["Jan Beulich" <JBeulich@xxxxxxxx>]

>>> On 26.06.15 at 14:37, <ian.campbell@xxxxxxxxxx> wrote:
> On Fri, 2015-06-26 at 12:16 +0100, Ian Campbell wrote:
>> On Fri, 2015-06-26 at 11:49 +0100, Jan Beulich wrote:
>> > >>> On 26.06.15 at 12:37, <ian.campbell@xxxxxxxxxx> wrote:
>> > > At Andy Cooper's request I ran a quick job with mtrr.show=true
>> > > http://logs.test-lab.xenproject.org/osstest/logs/58909/ 
>> > > 
>> > > I think the relevant serial output is:
>> > > Jun 26 09:57:42.325077 (XEN) MTRR default type: uncachable
>> > > Jun 26 09:57:42.325111 (XEN) MTRR fixed ranges enabled:
>> > > Jun 26 09:57:42.333068 (XEN)   00000-9ffff write-back
>> > > Jun 26 09:57:42.333101 (XEN)   a0000-bffff uncachable
>> > > Jun 26 09:57:42.333128 (XEN)   c0000-fffff write-back
>> > > Jun 26 09:57:42.341077 (XEN) MTRR variable ranges enabled:
>> > > Jun 26 09:57:42.341110 (XEN)   0 base 000000000000 mask ffff80000000 
> write-back
>> > > Jun 26 09:57:42.349088 (XEN)   1 base 000080000000 mask ffffc0000000 
> write-back
>> > > Jun 26 09:57:42.349124 (XEN)   2 disabled
>> > > Jun 26 09:57:42.357068 (XEN)   3 disabled
>> > > Jun 26 09:57:42.357098 (XEN)   4 disabled
>> > > Jun 26 09:57:42.357122 (XEN)   5 disabled
>> > > Jun 26 09:57:42.357147 (XEN)   6 disabled
>> > > Jun 26 09:57:42.365063 (XEN)   7 disabled
>> > 
>> > This alone would mean UC for all memory above 4G. But I seem to
>> > recall AMD having some mechanism to avoid using MTRRs for this
>> > case. Let me try to dig this out once back from lunch.
>> 
>> While you do that it seems like I may as well try a run with
>> "e820-mtrr-clip" given to Xen.
> 
> According to http://logs.test-lab.xenproject.org/osstest/logs/58914/ it
> didn't make any difference to the end result.
> 
> It did seems to cause a huge number of
> Jun 26 11:51:29.933067 (XEN) AMD-Vi: IO_PAGE_FAULT: domain = 0, device id = 
> 0x92, fault address = 0xbdfe7000, flags = 0
> messages which weren't there before, not sure if that is a clue or not.

I think that's a result of amd_iommu_hwdom_init() now stopping
below the reserved ranges right below 3Gb. I.e. these ought to
go away if you had the system use minimally more than 4Gb. I
also think that you'd see those too without limiting memory if the
reserved range was large enough to not share a PDX with the
highest RAM page below 4Gb (due to the way mfn_valid() works),
or if we indeed only mapped RAM pages there.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel