Re: [win-pv-devel] Advice on evtchn interrupt handling

[Paul Durrant <Paul.Durrant@xxxxxxxxxx>]

> -----Original Message-----
> From: win-pv-devel-bounces@xxxxxxxxxxxxxxxxxxxx [mailto:win-pv-devel-
> bounces@xxxxxxxxxxxxxxxxxxxx] On Behalf Of Rafal Wojdyla
> Sent: 09 December 2015 14:06
> To: Paul Durrant; win-pv-devel@xxxxxxxxxxxxxxxxxxxx
> Subject: Re: [win-pv-devel] Advice on evtchn interrupt handling
> 
> On 12/09/2015 02:02 PM, Paul Durrant wrote:
> >> -----Original Message-----
> >> From: RafaÅ WojdyÅa [mailto:omeg@xxxxxxxxxxxxxxxxxxxxxx]
> >> Sent: 09 December 2015 11:08
> >> To: Paul Durrant; win-pv-devel@xxxxxxxxxxxxxxxxxxxx
> >> Subject: Re: [win-pv-devel] Advice on evtchn interrupt handling
> >>
> >> On 2015-12-09 11:44, Paul Durrant wrote:
> >>>> -----Original Message-----
> >>>> From: win-pv-devel-bounces@xxxxxxxxxxxxxxxxxxxx [mailto:win-pv-
> devel-
> >>>> bounces@xxxxxxxxxxxxxxxxxxxx] On Behalf Of Rafal Wojdyla
> >>>> Sent: 09 December 2015 10:19
> >>>> To: win-pv-devel@xxxxxxxxxxxxxxxxxxxx
> >>>> Subject: [win-pv-devel] Advice on evtchn interrupt handling
> >>>>
> >>>> Hi,
> >>>>
> >>>> I found an issue in my evtchn handling in xeniface that I'm not sure how
> >>>> to properly fix. Namely, the interrupt callback that's assigned for a
> >>>> particular channel can be called before its queued DPC (that signals the
> >>>> caller-provided event) finishes. In such case the interrupt callback
> >>>> fails to insert the DPC again and an event is lost, potentially causing
> >>>> a vchan "deadlock" down the way.
> >>>>
> >>>
> >>> Hi Rafal,
> >>>
> >>> That shouldn't be the case. Windows de-queues a DPC before running
> for
> >> precisely this kind of reason. I.e. just because your DPC is running, it
> should
> >> not stop another from being queued. Are you definitely seeing a failure to
> >> queue even when no DPC is actually queued? What you may actually be
> >> getting caught by is evtchn auto-masking. If you have enabled auto-
> masking
> >> then you need to explicitly unmask at the end of your DPC. (The
> transmitter
> >> and receiver ring DPCs in XENVIF are a good example of this).
> >>>
> >>> Cheers,
> >>>
> >>>   Paul
> >>>
> >> Here's the relevant bit:
> >>
> http://xenbits.xen.org/gitweb/?p=pvdrivers/win/xeniface.git;a=blob;f=src/x
> >>
> eniface/ioctl_evtchn.c;h=9c5af19e00a91a08f68da0c630709cbaa6e8413d;hb=
> >> HEAD#l66
> >>
> >> I check the result of KeInsertQueueDpc() and I definitely see it fail
> >> sometimes, even if the HVM only has 1 vcpu. My DPC unmasks the
> channel
> >> after setting an event so this shouldn't be the reason.
> >>
> >
> > The article at https://www.osronline.com/article.cfm?article=529 is worth
> reading. The comment labelled "DPC Reentrancy" points out that DPCs are
> de-queued before being executed. So really you should not see a queueing
> failure unless either then unmasking is not working correctly or there's a bug
> in Windows itself.
> 
> Yeah, I've read that before and was wondering whether I'm missing
> something that would explain the behavior I see.

Actually, looking more closely at the code today I do see a problem...

You have a DPC per CPU so, if you have two separate event channels bound to the 
same CPU you're gonna hit problems. E.g. an event comes in on the first channel 
so you schedule a DPC to handle it. Now, an event comes in on the second 
channel and you can't schedule the DPC because itâs still queued for the first 
event. You need a DPC per channel.

  Paul

> 
> > Are you running on a Xen with FIFO events or is it old enough to only be 2L?
> >
> >   Paul
> 
> We're using 4.4.3 right now.
> 
> >
> >>>> Any advice on how to approach this? The interrupt callback runs at
> >>>> HIGH_LEVEL which is an issue when it comes to synchronization with
> >> some
> >>>> other code that could take care of such "pending" events.
> >>>>
> >>>> Also I wonder if the kernel differentiates DPCs only by their KDPC
> >>>> address, or also by arguments passed to KeInsertQueueDpc()?
> >>>>
> 
> --
> RafaÅ WojdyÅa
> Qubes Tools for Windows developer
> https://www.qubes-os.org/
> 
> _______________________________________________
> win-pv-devel mailing list
> win-pv-devel@xxxxxxxxxxxxxxxxxxxx
> http://lists.xenproject.org/cgi-bin/mailman/listinfo/win-pv-devel
_______________________________________________
win-pv-devel mailing list
win-pv-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/win-pv-devel