[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-devel] [PATCH] x86/pt: skip setup of posted format IRTE when gvec is 0
On Tue, Apr 30, 2019 at 05:48:14PM +0100, Andrew Cooper wrote: > On 30/04/2019 17:24, Chao Gao wrote: > > On Tue, Apr 30, 2019 at 11:08:54AM +0200, Roger Pau Monné wrote: > >> On Tue, Apr 30, 2019 at 01:19:19PM +0800, Chao Gao wrote: > >>> When testing with an UP guest with a pass-thru device with vt-d pi > >>> enabled in host, we observed that guest couldn't receive interrupts > >>> from that pass-thru device. Dumping IRTE, we found the corresponding > >>> IRTE is set to posted format with "vector" field as 0. > >>> > >>> We would fall into this issue when guest used the pirq format of MSI > >> I think the above sentence is a bit confusing. I would rather say that > >> the guest is routing interrupts from passthrough devices over event > >> channels using pirqs. > > Yes. It is better than it was. > > > >>> (see the comment xen_msi_compose_msg() in linux kernel). As 'dest_id' > >>> is repurposed, skip migration which is based on 'dest_id'. > >> Like Jan, I wonder why the device model (I assume QEMU) issues a > >> XEN_DOMCTL_bind_pt_irq hypercall when the guest is attempting to route > >> this interrupts over an event channel, attempting to bind it to a > >> vector seems like a bug in the device model. > >> > >> I would also consider whether it would make sense to not expose the > >> XENFEAT_hvm_pirqs feature when doing passthrough if posted interrupts > >> are supported, performance wise it's better to use posted interrupts > >> rather than routing them over event channels (which requires Xen > >> interaction). > > It is reasonable. But I think currently guest can add "xen_nopv" to its > > kernel boot options to avoid using evtchn. There might be some cases > > even with pass-thru devices (such as, guest uses polling mode > > driver for pass-thru devices), we care more about the efficiency of > > delivering virtual interrupts. So a separate option for evtchn in guest > > kernel seems like a better choice. > > This is a festering swamp, and is going to need some careful > architectural work to untangle. > > At the moment, guests which are xen-aware will try to use event channels > for everything. I'm pretty sure we've got some ABI incompatibilities > here with hardware APIC acceleration, and I still have yet(!) to > diagnose the underlying problem which is preventing XenServer from > enabling hardware APIC acceleration. (VMs still intermittently wedge on > migrate with what appears to be lost interrupt.) My opinion is that XENFEAT_hvm_pirqs should be removed: - It abuses of a hardware interface to pass Xen specific information. When routing pirqs over event channels for HVM the pirq is passed on the MSI address field, hijacking the native dest_id field. - Cannot be used with posted interrupts or APICV. Iff routing physical interrupts over event channels is still a useful thing for HVM, it should be done using hypercalls, like it's done on a PV dom0. > > The legacy HVM callback via single vector is incompatible with hardware > APIC acceleration, because it exists specifically to skip going through > the IRR. I can't think a proper solution to this, other than crashing > the guest when it tries to set such a situation up. > > From an "architecturally ideal" point of view, we obviously want to be > using APICV and/or Posted interrupt support whenever possible, and be > using these methods in preference to event channels. > > Injecting interrupts (including signalling an event upcall) using the > PIR is more efficient than other means, so long as we can ensure that > guest will handle the vector like any other edge triggered interrupt, > and ack it at the LAPIC. (This is where the legacy callback via method > breaks down, as it was intentionally designed to be used without an ack > at the LAPIC.) IIRC there's already a correct way to setup an event channel upcall against a vector using HVMOP_set_evtchn_upcall_vector which requires an APIC ack. Now the issue is switching existing users to this new callback setup mechanism. > As soon as we get to device interrupts, posted is definitely the way to > go, especially as the guest should not be aware of the routing behind > the scenes in the first place. > > To be clear, Chao - I'm not asking you to fix this. This is a decade of > technical debt which has been steadily growing, and it needs to start > with a coherent plan what the current state of play is, and how we'd > like to proceed. I agree. I've suggested a possible fix for the problem at hand, but as stated above I think XENFEAT_hvm_pirqs should be removed. Thanks, Roger. _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxxxxxxxxx https://lists.xenproject.org/mailman/listinfo/xen-devel
|
Lists.xenproject.org is hosted with RackSpace, monitoring our |