Re: NetBSD dom0 PVH: hardware interrupts stalls

[Manuel Bouyer <bouyer@xxxxxxxxxxxxxxx>]

On Fri, Nov 20, 2020 at 11:00:05AM +0100, Jan Beulich wrote:
> On 20.11.2020 10:27, Manuel Bouyer wrote:
> > On Fri, Nov 20, 2020 at 09:59:57AM +0100, Jan Beulich wrote:
> >> Well, anything coming through the LAPIC needs ack-ing (except for
> >> the spurious interrupt of course), or else ISR won't get updated
> >> and further interrupts at this or lower priority can't be serviced
> >> (delivered) anymore. This includes interrupts originally coming
> >> through the IO-APIC. But the same constraint / requirement exists
> >> on baremetal.
> > 
> > OK, so even if I didn't see where this happens, it's happening.
> > Is it what's Xen is using as ACK from the dom0 for a IOAPIC
> > interrupt, or is it something else (at the IOAPIC level) ?
> 
> It's the traditional LAPIC based EOI mechanism that Xen intercepts
> (as necessary) on the guest side and then translates (via
> surprisingly many layers of calls) into the necessary EOI /
> unmask / whatever at the hardware level. Our vIO-APIC
> implementation so far doesn't support IO-APIC based EOI at all
> (which is reflected in the IO-APIC version ID being 0x11).

OK.
I finally found where the EOI occurs (it's within a macro so s simple grep
didn't show it).

When interrupts are not masked (e.g. via cli), the ioapic halder is called.
>From here, 2 paths can happen:
a) the software interrupt priority level (called spl in BSD world) allows the
  driver's handler to run. In this case it's called, then the interrupt
  is unmasked at ioapic level, and EOI'd at lapic.
b) the software interrupt priority level doesn't allow this driver's handler to
  run. In this case, the interrupt is marked as pending in software,
  explicitely masked at the iopic level and EOI'd at lapic.
  Later, when the spl is lowered, the driver's interrupt handler is run,
  then the interrupt is unmasked at ioapic level, and EOI'd at lapic
  (this is the same path as a)). here we may EOI the lapic twice, and the
  second time when there's no hardware interrupt pending any more.

I suspect it's case b) which causes the problem with Xen.

-- 
Manuel Bouyer <bouyer@xxxxxxxxxxxxxxx>
     NetBSD: 26 ans d'experience feront toujours la difference
--