Re: [Xen-devel] VT-d Posted-interrupt (PI) design for XEN

["Wu, Feng" <feng.wu@xxxxxxxxx>]

> -----Original Message-----
> From: Tian, Kevin
> Sent: Tuesday, March 10, 2015 10:01 AM
> To: Andrew Cooper; Tim Deegan; Wu, Feng
> Cc: Zhang, Yang Z; Jan Beulich; xen-devel@xxxxxxxxxxxxx
> Subject: RE: [Xen-devel] VT-d Posted-interrupt (PI) design for XEN
> 
> > From: Andrew Cooper [mailto:andrew.cooper3@xxxxxxxxxx]
> > Sent: Monday, March 09, 2015 7:46 PM
> >
> > On 09/03/15 10:33, Tim Deegan wrote:
> > > At 02:03 +0000 on 09 Mar (1425863009), Wu, Feng wrote:
> > >>
> > >>> -----Original Message-----
> > >>> From: Tim Deegan [mailto:tim@xxxxxxx]
> > >>> Sent: Friday, March 06, 2015 5:44 PM
> > >>> To: Wu, Feng
> > >>> Cc: Jan Beulich; Zhang, Yang Z; Tian, Kevin; xen-devel@xxxxxxxxxxxxx
> > >>> Subject: Re: [Xen-devel] VT-d Posted-interrupt (PI) design for XEN
> > >>>
> > >>> At 02:07 +0000 on 06 Mar (1425604054), Wu, Feng wrote:
> > >>>>> From: Tim Deegan [mailto:tim@xxxxxxx]
> > >>>>> But I don't understand why we would need a new global vector for
> > >>>>> RUNSTATE_blocked rather than suppressing the posted interrupts as
> you
> > >>>>> suggest for RUNSTATE_runnable.  (Or conversely why not use the new
> > >>>>> global vector for RUNSTATE_runnable too?)
> > >>>> If we suppress the posted-interrupts when vCPU is blocked, it cannot
> > >>>> be unblocked by the external interrupts, this is not correct.
> > >>> OK, I don't understand at all now. :)  When the posted interrupt is
> > >>> suppressed, what happens to the interrupt?
> > >> When the posted interrupt is suppressed, VT-d engine will not issue
> > >> notification events.
> > >>
> > >>> If it's just dropped, then we can't use that for _any_ cases.
> > >> We can suppress the posted-interrupt when vCPU is waiting in the
> runqueue
> > >> (vCPU is in RUNSTATE_runnable state), it is not needed to send 
> > >> notification
> > >> event when vCPU is in this state, since when interrupt happens, the
> > interrupt
> > >> information are not _dropped_, instead, they are stored in PIR, and this
> will
> > >> be synced to vIRR before VM-Entry.
> > > So you think you can use the same system for RUNSTATE_runnable as
> > > RUNSTATE_blocked?  That seems like a good idea.
> > >
> > > I'll leave the details (e.g. single global vector + queue vs any other
> > > way to wake the vcpu) to people who know the x86 irq code better than
> > > I do. :)
> >
> > From my reading the relevant section in the VT-d spec, to the best of my
> > understanding:
> >
> > We only need the second vector if Xen wishes to be informed that an
> > interrupt has been queued for a vcpu.  The spec suggests that, for one
> > usecase, this information should affect scheduling decisions.
> >
> > If we do not wish to make scheduling alterations based on interrupt
> > delivery, the extra vector can be ignored.
> >
> > If we do wish to make scheduling alterations, we will need to be able to
> > uniquely identify a vcpu from a vector, which will involve allocating
> > one vector per vcpu.
> >
> >
> > If my understanding is correct, I would suggest that Xen opt for not
> > getting notifications.  Interrupting one guest to indicate that another
> > vcpu has been interrupted scales progressively worse with the number of
> > running VMs, and there are existing usecases which have already
> > exhausted the x86 vector space completely.
> >
> > It might be sensible to have the option available as a per-domain opt-in
> > option.  A usecase such as device driver domain could easily want to
> > deal with its interrupts ahead of running the domains it is servicing.
> >
> 
> IMO we don't need such opt. An blocked VCPU may not be woken up
> when losing a virtual interrupt notification, and if you look at earlier
> reply to Jan it's not necessarily to have one-vector-per-vcpu. It's just
> a global vector, which when sent to a specific pcpu, the handler will
> walk through blocked vcpus on that pcpu to decide which one should
> be woken up. So only one new vector is required.
> 
> from Feng's design, the notification may be disabled in one scenario,
> i.e. when vcpu is in runnable state. That works if real-time is not
> considered since we know runnable vcpu is already unblocked. Later
> when considering real-time, this notification will be required too.

Thanks for your clarification, Kevin!

Thanks,
Feng

> 
> Thanks
> Kevin

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel