Re: [Xen-users] (3ware) xen 3.0 amd64 crash... seems to be tied into disk i/o,> 4 gig ram

[Tom Brown <tbrown@xxxxxxxxxxxxx>]

On Fri, 9 Dec 2005, Matt Ayres wrote:

> Tom Brown wrote:
> > On Thu, 8 Dec 2005, Ian Pratt wrote:
> >
> >> Looking at thhe oops message, this is with a 3ware card, right?
> >> We've had at least one other report of them causing problems on systems
> >> with >4GB enabled (or maybe it was you?)
> >
> > Yes, I am (or was) using a 3ware controller, since the on board sata
> > controller is a marvell one, and the native linux drivers seem flaky as
> > can be. I've only had the 3ware card for about a week, and don't believe I
> > have written about it. I was going to ask on this list about the MCE,
> > since I don't know of a better list for asking questions about big opteron
> > systems.
>
> I was the other who reported (via bugzilla, #402) a crash with 3ware.  I
> was told the 3ware driver does not look >4GB safe.  I run the same
> hardware specs using other (non-xen) kernels using PAE and >4GB and have
> absolutely no troubles though.  I am still in a re-testing phase since
> 3.0.0 was announced.  I have yet to put customers on this setup (which
> is where whatever can happen, will happen).

I got a response back from 3ware, as I forwarded Ian's note to them...
My card(s) is(are) an 8506-4LP

I don't know much about 'the use of the IOMMU'. I do know that at
least one of my kernels was bitching about the bios settings for
IOMMU, and it seems to be an active area of Linux development. It
seems to be some sort of aperature possibly used as a temporary
storage space for information destined for addresses > 4gig? It
also appears to tied into the onboard (northbridge) GART stuff. I
believe the engineers MCE/memory comments apply to the controller
memory, as I would expect a clearer MCE message if we got an ECC
fault on main memory.

If anyone can recommed a good "big linux", or amd64 linux list,
I'd be happier posting this stuff there... although I expect
there are more "high powered techs" on this list, than most
others... of course there are a lot of "very junior" sys admins
here too :-)

   Date: Thu, 8 Dec 2005 14:32:58 -0800
   From: David Graas <@amcc.com>
   To: Tom Brown <tbrown@xxxxxxxxxxxxx>
   Subject: RE: [Xen-users] xen 3.0 amd64 crash... seems to be tied into disk
       i/o, > 4 gig ram (fwd)

   Tom, I am not sure if this helps or not but your e-mail did concern me
   so I checked with some of our engineers on this. Their reply is below...

   "We have supported > 4GB ram, even with the 3w-xxxx driver and
   5000/6000/7000/8000 series cards on amd64 through the use of the IOMMU
   GART (Northbridge memory aperature) before the Opteron even shipped to
   customers, even though these cards only do 32-bit DMA operations.

   We have supported > 4GB ram with the 9500-S and 9550-SX since day one.
   We also support PAE (Physical Address Extensions) mode with the 9500-S
   and 9550-SX So you can DMA to > 4GB of ram on a 32-bit pentium pro or
   higher architecture.

   Also, he is getting MCE error messages (Machine Check Exceptions) from
   his Northbridge, which could mean bad memory."

   It might be worth a call to our support group (800-840-6055) to see if
   there is a fix on this situation.


   David Graas
   Corporate Sales Manager
   3ware - an AMCC company
   Direct 408-542-8670
   Mobile 650-269-2972
   Fax 408-542-8602
   @amcc.com
   www.amcc.com


I've invalidated David's address, but it is first inital, lastname if
you want to write to him...


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users