Re: [win-pv-devel] Advice on evtchn interrupt handling

[RafaÅ WojdyÅa <omeg@xxxxxxxxxxxxxxxxxxxxxx>]

On 2015-12-10 13:27, Paul Durrant wrote:
>> -----Original Message-----
>> From: win-pv-devel-bounces@xxxxxxxxxxxxxxxxxxxx [mailto:win-pv-devel-
>> bounces@xxxxxxxxxxxxxxxxxxxx] On Behalf Of Rafal Wojdyla
>> Sent: 09 December 2015 14:06
>> To: Paul Durrant; win-pv-devel@xxxxxxxxxxxxxxxxxxxx
>> Subject: Re: [win-pv-devel] Advice on evtchn interrupt handling
>>
>> On 12/09/2015 02:02 PM, Paul Durrant wrote:
>>>> -----Original Message-----
>>>> From: RafaÅ WojdyÅa [mailto:omeg@xxxxxxxxxxxxxxxxxxxxxx]
>>>> Sent: 09 December 2015 11:08
>>>> To: Paul Durrant; win-pv-devel@xxxxxxxxxxxxxxxxxxxx
>>>> Subject: Re: [win-pv-devel] Advice on evtchn interrupt handling
>>>>
>>>> On 2015-12-09 11:44, Paul Durrant wrote:
>>>>>> -----Original Message-----
>>>>>> From: win-pv-devel-bounces@xxxxxxxxxxxxxxxxxxxx [mailto:win-pv-
>> devel-
>>>>>> bounces@xxxxxxxxxxxxxxxxxxxx] On Behalf Of Rafal Wojdyla
>>>>>> Sent: 09 December 2015 10:19
>>>>>> To: win-pv-devel@xxxxxxxxxxxxxxxxxxxx
>>>>>> Subject: [win-pv-devel] Advice on evtchn interrupt handling
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I found an issue in my evtchn handling in xeniface that I'm not sure how
>>>>>> to properly fix. Namely, the interrupt callback that's assigned for a
>>>>>> particular channel can be called before its queued DPC (that signals the
>>>>>> caller-provided event) finishes. In such case the interrupt callback
>>>>>> fails to insert the DPC again and an event is lost, potentially causing
>>>>>> a vchan "deadlock" down the way.
>>>>>>
>>>>>
>>>>> Hi Rafal,
>>>>>
>>>>> That shouldn't be the case. Windows de-queues a DPC before running
>> for
>>>> precisely this kind of reason. I.e. just because your DPC is running, it
>> should
>>>> not stop another from being queued. Are you definitely seeing a failure to
>>>> queue even when no DPC is actually queued? What you may actually be
>>>> getting caught by is evtchn auto-masking. If you have enabled auto-
>> masking
>>>> then you need to explicitly unmask at the end of your DPC. (The
>> transmitter
>>>> and receiver ring DPCs in XENVIF are a good example of this).
>>>>>
>>>>> Cheers,
>>>>>
>>>>>   Paul
>>>>>
>>>> Here's the relevant bit:
>>>>
>> http://xenbits.xen.org/gitweb/?p=pvdrivers/win/xeniface.git;a=blob;f=src/x
>>>>
>> eniface/ioctl_evtchn.c;h=9c5af19e00a91a08f68da0c630709cbaa6e8413d;hb=
>>>> HEAD#l66
>>>>
>>>> I check the result of KeInsertQueueDpc() and I definitely see it fail
>>>> sometimes, even if the HVM only has 1 vcpu. My DPC unmasks the
>> channel
>>>> after setting an event so this shouldn't be the reason.
>>>>
>>>
>>> The article at https://www.osronline.com/article.cfm?article=529 is worth
>> reading. The comment labelled "DPC Reentrancy" points out that DPCs are
>> de-queued before being executed. So really you should not see a queueing
>> failure unless either then unmasking is not working correctly or there's a 
>> bug
>> in Windows itself.
>>
>> Yeah, I've read that before and was wondering whether I'm missing
>> something that would explain the behavior I see.
> 
> Actually, looking more closely at the code today I do see a problem...
> 
> You have a DPC per CPU so, if you have two separate event channels bound to 
> the same CPU you're gonna hit problems. E.g. an event comes in on the first 
> channel so you schedule a DPC to handle it. Now, an event comes in on the 
> second channel and you can't schedule the DPC because itâs still queued for 
> the first event. You need a DPC per channel.
> 
>   Paul
> 
Ha, that's exactly what I found as well. I implemented a similar change
yesterday and everything seemed fine during my tests. Thanks for the
confirmation - I think you can apply the patch.

>>
>>> Are you running on a Xen with FIFO events or is it old enough to only be 2L?
>>>
>>>   Paul
>>
>> We're using 4.4.3 right now.
>>
>>>
>>>>>> Any advice on how to approach this? The interrupt callback runs at
>>>>>> HIGH_LEVEL which is an issue when it comes to synchronization with
>>>> some
>>>>>> other code that could take care of such "pending" events.
>>>>>>
>>>>>> Also I wonder if the kernel differentiates DPCs only by their KDPC
>>>>>> address, or also by arguments passed to KeInsertQueueDpc()?
>>>>>>

-- 
RafaÅ WojdyÅa
Qubes Tools for Windows developer
https://www.qubes-os.org/

_______________________________________________
win-pv-devel mailing list
win-pv-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/win-pv-devel