Re: NetBSD dom0 PVH: hardware interrupts stalls

[Roger Pau Monné <roger.pau@xxxxxxxxxx>]

On Wed, Nov 18, 2020 at 10:24:25AM +0100, Manuel Bouyer wrote:
> On Wed, Nov 18, 2020 at 09:57:38AM +0100, Roger Pau Monné wrote:
> > On Tue, Nov 17, 2020 at 05:40:33PM +0100, Manuel Bouyer wrote:
> > > On Tue, Nov 17, 2020 at 04:58:07PM +0100, Roger Pau Monné wrote:
> > > > [...]
> > > > 
> > > > I have attached a patch below that will dump the vIO-APIC info as part
> > > > of the 'i' debug key output, can you paste the whole output of the 'i'
> > > > debug key when the system stalls?
> > > 
> > > see attached file. Note that the kernel did unstall while 'i' output was
> > > being printed, so it is mixed with some NetBSD kernel output.
> > > The idt entry of the 'ioapic2 pin2' interrupt is 103 on CPU 0.
> > > 
> > > I also put the whole sequence at
> > > http://www-soc.lip6.fr/~bouyer/xen-log3.txt
> > 
> > On one of the instances the pin shows up as masked, but I'm not sure
> > if that's relevant since later it shows up as unmasked. Might just be
> > part of how NetBSD handles such interrupts.
> 
> Yes, NetBSD can mask an interrupt source if the interrupts needs to be 
> delayed.
> It will be unmasked once the interrupt has been handled.

Yes, I think that's roughly the same model that FreeBSD uses for
level IO-APIC interrupts: mask it until the handlers have been run.

> Would it be possible that Xen misses an unmask write, or fails to
> call the vector if the interrupt is again pending at the time of the
> unmask ?

Well, it should work properly, but we cannot discard anything.

> > [...]
> > On a maybe unrelated question, how do you setup the event channel
> > callback, is it using HVM_PARAM_CALLBACK_IRQ and
> > HVM_PARAM_CALLBACK_TYPE_VECTOR?
> 
> Yes, the code is at
> https://github.com/NetBSD/src/blob/f9a54eaecfb47bce597f72f6cae8861f4d486eb4/sys/arch/xen/xen/hypervisor.c#L457
> 
> > 
> > Are you EOI'ing such vector on the local APIC when servicing the
> > interrupt?
> 
> I think it's OK. the code is at
> https://github.com/NetBSD/src/blob/f9a54eaecfb47bce597f72f6cae8861f4d486eb4/sys/arch/amd64/amd64/vector.S#L770

Yes, it's fine as you also have support for the newly per-CPU vector
callback which does require an EOI.

Roger.