Re: [Xen-devel] (v2) VT-d Posted-interrupt (PI) design for XEN

["Wu, Feng" <feng.wu@xxxxxxxxx>]

> -----Original Message-----
> From: Tian, Kevin
> Sent: Tuesday, March 24, 2015 11:06 AM
> To: Wu, Feng; Jan Beulich
> Cc: Zhang, Yang Z; xen-devel@xxxxxxxxxxxxx; Keir Fraser (keir@xxxxxxx)
> Subject: RE: (v2) VT-d Posted-interrupt (PI) design for XEN
> 
> > From: Wu, Feng
> > Sent: Monday, March 23, 2015 5:19 PM
> >
> >
> >
> > > -----Original Message-----
> > > From: Jan Beulich [mailto:JBeulich@xxxxxxxx]
> > > Sent: Monday, March 23, 2015 5:08 PM
> > > To: Wu, Feng
> > > Cc: Tian, Kevin; Zhang, Yang Z; xen-devel@xxxxxxxxxxxxx; Keir Fraser
> > > (keir@xxxxxxx)
> > > Subject: RE: (v2) VT-d Posted-interrupt (PI) design for XEN
> > >
> > > >>> On 23.03.15 at 09:49, <feng.wu@xxxxxxxxx> wrote:
> > >
> > > >
> > > >> -----Original Message-----
> > > >> From: Jan Beulich [mailto:JBeulich@xxxxxxxx]
> > > >> Sent: Monday, March 23, 2015 4:26 PM
> > > >> To: Wu, Feng
> > > >> Cc: Tian, Kevin; Zhang, Yang Z; xen-devel@xxxxxxxxxxxxx; Keir Fraser
> > > >> (keir@xxxxxxx)
> > > >> Subject: RE: (v2) VT-d Posted-interrupt (PI) design for XEN
> > > >>
> > > >> >>> On 23.03.15 at 09:14, <feng.wu@xxxxxxxxx> wrote:
> > > >> >> From: Jan Beulich [mailto:JBeulich@xxxxxxxx]
> > > >> >> Sent: Thursday, March 19, 2015 5:57 PM
> > > >> >> >>> On 18.03.15 at 13:44, <feng.wu@xxxxxxxxx> wrote:
> > > >> >> > Here are what we do for the blocked vCPU:
> > > >> >> > 1. Define a per-cpu list 'blocked_vcpu_on_cpu', which stored the
> > > blocked
> > > >> >> > vCPU on the pCPU.
> > > >> >> > 2. When the vCPU's state is changed to RUNSTATE_blocked, insert
> > the
> > > >> vCPU
> > > >> >> > to the per-cpu list belonging to the pCPU it was running.
> > > >> >> > 3. When the vCPU is unblocked, remove the vCPU from the related
> > > pCPU
> > > >> list.
> > > >> >>
> > > >> >> And this works transparently not only with the generic scheduler
> > > >> >> code moving the vCPU to another pCPU, but also with some of the
> > > >> >> individual scheduler implementations doing such re-assignments?
> > > >> >
> > > >> > I cannot quite understand this, could you please elaborate a bit 
> > > >> > more.
> > > >>
> > > >> There are multiple places where v->processor can get changed for a
> > > >> particular vCPU, and obviously all of these need to be taken care of.
> > > >> Yet a change like the one to come here would normally not be
> > > >> expected to touch specific schedulers' code, and hence suitably
> > > >> abstracting this may need some extra thought.
> > > >
> > > > Why do we need care about the places where v->processor gets changed,
> > > > my idea about this is:
> > > >
> > > > Before vCPU is blocked, we can get v->processor, and save the vCPU to
> > > > this per-CPU list. Besides that v->processor is the destination of the
> > > > notification
> > > > event (it is stored in Posted-interrupt descriptor). So when wakeup
> > > > notification event happens form this vCPU, it goes to pCPU v->processor,
> > > > then in the wakeup notification event handler, we can find the list via
> > > > smp_processor_id(), hence find the right vCPU to wake up.
> > > >
> > > > Do I miss something here?
> > >
> > > Perhaps you don't, and perhaps I implied things I shouldn't have
> > > implied: When v->processor changes, it would look to me that the
> > > respective vCPU then ends up on the wrong list. If that's not a
> > > problem - fine.
> >
> > Yes, vCPU is not changed to another list when v->processor is changed.
> >
> > Using per-CPU lists, however, would seem to make
> > > it desirable to access those lists without lock, yet that can't work
> > > when the list may get accessed from other than the owning CPU.
> >
> > Yes, it need to be accessed in other CPUs.
> >
> 
> Then you do need some lock mechanism to avoid race condition. Possibly
> it's not a big problem if all the operations around this per-cpu list are
> associated with scheduling logic so we might leverage the scheduling lock
> for the purpose.
> 

Thanks for the comments, Kevin!

Yes, we need spin lock to protect the list. Besides the scheduling logic, we
also need to access the list in the wakeup notification event handler, which
is in interrupt context.

Thanks,
Feng

> Thanks
> Kevin

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel