Re: [Xen-devel] VT-d Posted-interrupt (PI) design for XEN

["Tian, Kevin" <kevin.tian@xxxxxxxxx>]

> From: Andrew Cooper [mailto:andrew.cooper3@xxxxxxxxxx]
> Sent: Monday, March 09, 2015 7:46 PM
> 
> On 09/03/15 10:33, Tim Deegan wrote:
> > At 02:03 +0000 on 09 Mar (1425863009), Wu, Feng wrote:
> >>
> >>> -----Original Message-----
> >>> From: Tim Deegan [mailto:tim@xxxxxxx]
> >>> Sent: Friday, March 06, 2015 5:44 PM
> >>> To: Wu, Feng
> >>> Cc: Jan Beulich; Zhang, Yang Z; Tian, Kevin; xen-devel@xxxxxxxxxxxxx
> >>> Subject: Re: [Xen-devel] VT-d Posted-interrupt (PI) design for XEN
> >>>
> >>> At 02:07 +0000 on 06 Mar (1425604054), Wu, Feng wrote:
> >>>>> From: Tim Deegan [mailto:tim@xxxxxxx]
> >>>>> But I don't understand why we would need a new global vector for
> >>>>> RUNSTATE_blocked rather than suppressing the posted interrupts as you
> >>>>> suggest for RUNSTATE_runnable.  (Or conversely why not use the new
> >>>>> global vector for RUNSTATE_runnable too?)
> >>>> If we suppress the posted-interrupts when vCPU is blocked, it cannot
> >>>> be unblocked by the external interrupts, this is not correct.
> >>> OK, I don't understand at all now. :)  When the posted interrupt is
> >>> suppressed, what happens to the interrupt?
> >> When the posted interrupt is suppressed, VT-d engine will not issue
> >> notification events.
> >>
> >>> If it's just dropped, then we can't use that for _any_ cases.
> >> We can suppress the posted-interrupt when vCPU is waiting in the runqueue
> >> (vCPU is in RUNSTATE_runnable state), it is not needed to send notification
> >> event when vCPU is in this state, since when interrupt happens, the
> interrupt
> >> information are not _dropped_, instead, they are stored in PIR, and this 
> >> will
> >> be synced to vIRR before VM-Entry.
> > So you think you can use the same system for RUNSTATE_runnable as
> > RUNSTATE_blocked?  That seems like a good idea.
> >
> > I'll leave the details (e.g. single global vector + queue vs any other
> > way to wake the vcpu) to people who know the x86 irq code better than
> > I do. :)
> 
> From my reading the relevant section in the VT-d spec, to the best of my
> understanding:
> 
> We only need the second vector if Xen wishes to be informed that an
> interrupt has been queued for a vcpu.  The spec suggests that, for one
> usecase, this information should affect scheduling decisions.
> 
> If we do not wish to make scheduling alterations based on interrupt
> delivery, the extra vector can be ignored.
> 
> If we do wish to make scheduling alterations, we will need to be able to
> uniquely identify a vcpu from a vector, which will involve allocating
> one vector per vcpu.
> 
> 
> If my understanding is correct, I would suggest that Xen opt for not
> getting notifications.  Interrupting one guest to indicate that another
> vcpu has been interrupted scales progressively worse with the number of
> running VMs, and there are existing usecases which have already
> exhausted the x86 vector space completely.
> 
> It might be sensible to have the option available as a per-domain opt-in
> option.  A usecase such as device driver domain could easily want to
> deal with its interrupts ahead of running the domains it is servicing.
> 

IMO we don't need such opt. An blocked VCPU may not be woken up
when losing a virtual interrupt notification, and if you look at earlier
reply to Jan it's not necessarily to have one-vector-per-vcpu. It's just
a global vector, which when sent to a specific pcpu, the handler will
walk through blocked vcpus on that pcpu to decide which one should
be woken up. So only one new vector is required.

from Feng's design, the notification may be disabled in one scenario,
i.e. when vcpu is in runnable state. That works if real-time is not
considered since we know runnable vcpu is already unblocked. Later
when considering real-time, this notification will be required too.

Thanks
Kevin

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel