[Xen-devel] Re: [PATCH 08/13] xen/pvticketlock: disable interrupts while blocking

[Don Zickus <dzickus@xxxxxxxxxx>]

On Wed, Sep 07, 2011 at 07:13:58AM +0300, Avi Kivity wrote:
> On 09/06/2011 09:27 PM, Don Zickus wrote:
> >On Tue, Sep 06, 2011 at 11:07:26AM -0700, Jeremy Fitzhardinge wrote:
> >>  >>  But, erm, does that even make sense?  I'm assuming the NMI reason port
> >>  >>  tells the CPU why it got an NMI.  If multiple CPUs can get NMIs and
> >>  >>  there's only a single reason port, then doesn't that mean that either 
> >> 1)
> >>  >>  they all got the NMI for the same reason, or 2) having a single port 
> >> is
> >>  >>  inherently racy?  How does the locking actually work there?
> >>  >  The reason port is for an external/system NMI.  All the IPI-NMI don't 
> >> need
> >>  >  to access this register to process their handlers, ie perf.  I think in
> >>  >  general the IOAPIC is configured to deliver the external NMI to one 
> >> cpu,
> >>  >  usually the bsp cpu.  However, there has been a slow movement to free 
> >> the
> >>  >  bsp cpu from exceptions like this to allow one to eventually hot-swap 
> >> the
> >>  >  bsp cpu.  The spin locks in that code were an attempt to be more 
> >> abstract
> >>  >  about who really gets the external NMI.  Of course SGI's box is setup 
> >> to
> >>  >  deliver an external NMI to all cpus to dump the stack when the system
> >>  >  isn't behaving.
> >>  >
> >>  >  This is a very low usage NMI (in fact almost all cases lead to loud
> >>  >  console messages).
> >>  >
> >>  >  Hope that clears up some of the confusion.
> >>
> >>  Hm, not really.
> >>
> >>  What does it mean if two CPUs go down that path?  Should one do some NMI
> >>  processing while the other waits around for it to finish, and then do
> >>  some NMI processing on its own?
> >
> >Well the time the second one gets to the external NMI it should have been
> >cleared by the first cpu, which would of course lead to the second cpu
> >causing a 'Dazed and confused' message.  But on most x86 machines only one
> >cpu should be routed the external NMI.  Though there is an SGI box that is
> >designed to send an external NMI to all of its cpus.
> 
> Is there a way to tell whether an NMI was internally or externally
> generated?
> 
> I don't think so, especially as two or more NMIs can be coalesced.
> So any NMI received on this first cpu has to check the NMI reason
> port?

Well we cheat and execute all the nmi handlers first.  If they come back
as handled, we skip the check for the external NMI.

But you are right, other than checking the reason port, there isn't a way
to determine if an NMI is internally or externally generated.

> 
> >>
> >>  But on the other hand, I don't really care if you can say that this path
> >>  will never be called in a virtual machine.
> >
> >Does virtual machines support hot remove of cpus?  Probably not
> >considering bare-metal barely supports it.
> >
> 
> They do.

But vcpus probably don't have the notion of a bsp cpu, so perhaps virtual
machines can get away with it easier?  (I don't know enough about the hot
cpu remove code to really explain it, just enough to know it can cause
problems and people are trying to address it).

Cheers,
Don

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel