From win-pv-devel-bounces@lists.xenproject.org Mon Mar 04 07:12:05 2024
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Mon, 04 Mar 2024 07:12:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.688147.1072000 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rh2UU-0000g8-Iw; Mon, 04 Mar 2024 07:12:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 688147.1072000; Mon, 04 Mar 2024 07:12:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rh2UU-0000g1-GR; Mon, 04 Mar 2024 07:12:02 +0000
Received: by outflank-mailman (input) for mailman id 688147;
 Mon, 04 Mar 2024 07:12:00 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=y0FV=KK=cloud.com=owen.smith@srs-se1.protection.inumbo.net>)
 id 1rh2US-0000fv-TL
 for win-pv-devel@lists.xenproject.org; Mon, 04 Mar 2024 07:12:00 +0000
Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com
 [2a00:1450:4864:20::32a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7c4862e7-d9f6-11ee-a1ee-f123f15fe8a2;
 Mon, 04 Mar 2024 08:11:58 +0100 (CET)
Received: by mail-wm1-x32a.google.com with SMTP id
 5b1f17b1804b1-412a9e9c776so29757775e9.0
 for <win-pv-devel@lists.xenproject.org>; Sun, 03 Mar 2024 23:11:56 -0800 (PST)
Received: from localhost.localdomain
 (host86-169-131-93.range86-169.btcentralplus.com. [86.169.131.93])
 by smtp.gmail.com with ESMTPSA id
 m17-20020a05600c4f5100b00412b7456ef7sm13897117wmq.5.2024.03.03.23.11.55
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Sun, 03 Mar 2024 23:11:55 -0800 (PST)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7c4862e7-d9f6-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1709536316; x=1710141116; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=ZA++BEqZy2TutrZ5vy23omQ7GMGV+MUkO2LknVmsyAM=;
        b=JhC6eeMY9WUlSLSw7nFDRpiwJZtLx71cG7vPCm2I5qhgK7uRs+9r4qGxaNoTG7g3Yf
         6YFWmok4OoVsBnJsQQ/v79xjzeacngIecV3YBHKVhQcN6Qy6EbX0EaaGLPiRoLObl/VJ
         Xyqgm0oNekTEx9jplGssfMit0Lns8cQuCdVgc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709536316; x=1710141116;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=ZA++BEqZy2TutrZ5vy23omQ7GMGV+MUkO2LknVmsyAM=;
        b=PGzgvYxKLjL3Tdqsbl9GMRtAnEgsCBIRLhfynNfYF7dyuh8DivGYIigglfaLLrGjQh
         ybq6RfVaabzafyHB1VgAUutMmw/opdXKdYwElcTBK+CASLbqBAgrcQaXqiufg11ZXzr2
         4AvYBXgHCFkY9aHFBmoi+R1t1itwz955XrZOKA+Zr19orkk2mhy2Qmg8IdGRDm2XNSB+
         /SVo4TpE0QDapyqfpd0dG1r1c83EFEEt/e/JhBRiUADD9jyDwfgQ5vAwbZSKP6z9HQCh
         0FmhNpKbOGwOI48z9nwMb/IUR4Dh3QIVcgvW5Lb83osqXxYIZKKyx3jp4Tp31oefvoQU
         cI4g==
X-Gm-Message-State: AOJu0Yw6G4ki8h4T8amAG4Xgf6hsZiCnFEJN679rhf+JH0/LG9yK5mF6
	PSKWEeIQ4LSBb2sbBAw81SqXR3TOaz70ZwpHi3pnl87mxfJ3eZz+WeaZ0CUNg5ttbyeTucMBEQk
	4PQ==
X-Google-Smtp-Source: AGHT+IEdc6c3cz1rnLBJdsnwlGz3rNpjgnQP28qU1vrqR1TacKB11QL0+bOOk7EsbAXFixmJdlXV5w==
X-Received: by 2002:a05:600c:3108:b0:412:bf3b:ceea with SMTP id g8-20020a05600c310800b00412bf3bceeamr8423202wmo.16.1709536315870;
        Sun, 03 Mar 2024 23:11:55 -0800 (PST)
From: Owen Smith <owen.smith@cloud.com>
To: win-pv-devel@lists.xenproject.org
Cc: Owen Smith <owen.smith@cloud.com>
Subject: [PATCH] Adjust Packet's MDL when stripping ETHERNET_TAG from headers
Date: Mon,  4 Mar 2024 07:11:29 +0000
Message-ID: <20240304071129.2-1-owen.smith@cloud.com>
X-Mailer: git-send-email 2.41.0.windows.3
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Tagged headers have the ETHERNET_TAG stripped to convert to
ETHERNET_UNTAGGED_HEADERs. As part of the fix up, the offsets for each
header was adjusted, and the underlying MDL's MappedSystemVa was adjusted.
However, the ByteCount and ByteOffset were not adjusted, resulting in
instances of data corruption when MappedSystemVa != StartVa + ByteOffset.

It is possible to configure the default gateway to respond to ARP packets
with TPID wrappings. Openvswitch will not strip this header since this commit:

    commit f0fb825a3785320430686834741c718ff4f8ebf4
    Author: Eric Garver <e@erig.me>
    Date:   Wed Mar 1 17:47:59 2017 -0500

        Add support for 802.1ad (QinQ tunneling)

Which means Windows was receiving corrupted ARP responses, leading to an
inability to access the network correctly. This would usually show as
Windows indicating that no internet access was available, or being unable
it access other VMs or other resources beyond the default gateway.

Signed-off-by: Owen Smith <owen.smith@cloud.com>
---
 src/xenvif/receiver.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/src/xenvif/receiver.c b/src/xenvif/receiver.c
index 0ce5b67..720d1d1 100644
--- a/src/xenvif/receiver.c
+++ b/src/xenvif/receiver.c
@@ -436,16 +436,23 @@ ReceiverRingProcessTag(
 
     Packet->TagControlInformation = NTOHS(EthernetHeader->Tagged.Tag.ControlInformation);
 
+    // Convert ETHERNET_TAGGED_HEADER into ETHERNET_UNTAGGED_HEADER
     Offset = FIELD_OFFSET(ETHERNET_TAGGED_HEADER, Tag);
     RtlMoveMemory((PUCHAR)EthernetHeader + sizeof (ETHERNET_TAG),
                   (PUCHAR)EthernetHeader,
                   Offset);
 
     // Fix up the packet information
+    BaseVa = Packet->Mdl.MappedSystemVa;
+    ASSERT(BaseVa != NULL);
+
     BaseVa += sizeof (ETHERNET_TAG);
 
-    BaseVa -= Packet->Offset;
     Packet->Mdl.MappedSystemVa = BaseVa;
+    Packet->Mdl.ByteOffset += sizeof (ETHERNET_TAG);
+    Packet->Mdl.ByteCount -= sizeof (ETHERNET_TAG);
+
+    ASSERT3P((PUCHAR)Packet->Mdl.StartVa + Packet->Mdl.ByteOffset, ==, Packet->Mdl.MappedSystemVa);
 
     Packet->Length -= sizeof (ETHERNET_TAG);
 
@@ -468,10 +475,6 @@ ReceiverRingProcessTag(
 
     Info->Length -= sizeof (ETHERNET_TAG);
 
-    BaseVa += Packet->Offset;
-
-    EthernetHeader = (PETHERNET_HEADER)(BaseVa + Info->EthernetHeader.Offset);
-
     ASSERT3U(PayloadLength, ==, Packet->Length - Info->Length);
 }
 
-- 
2.41.0.windows.3



From win-pv-devel-bounces@lists.xenproject.org Tue Mar 05 09:42:42 2024
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Tue, 05 Mar 2024 09:42:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.688665.1073123 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rhRJn-00063z-RH; Tue, 05 Mar 2024 09:42:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 688665.1073123; Tue, 05 Mar 2024 09:42:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rhRJn-00063s-Ok; Tue, 05 Mar 2024 09:42:39 +0000
Received: by outflank-mailman (input) for mailman id 688665;
 Tue, 05 Mar 2024 09:42:37 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=HlLy=KL=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1rhRJl-00063m-Tb
 for win-pv-devel@lists.xenproject.org; Tue, 05 Mar 2024 09:42:37 +0000
Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com
 [2a00:1450:4864:20::434])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b2302060-dad4-11ee-a1ee-f123f15fe8a2;
 Tue, 05 Mar 2024 10:42:36 +0100 (CET)
Received: by mail-wr1-x434.google.com with SMTP id
 ffacd0b85a97d-33e285a33bdso1871804f8f.2
 for <win-pv-devel@lists.xenproject.org>; Tue, 05 Mar 2024 01:42:35 -0800 (PST)
Received: from [192.168.19.5] (54-240-197-238.amazon.com. [54.240.197.238])
 by smtp.gmail.com with ESMTPSA id
 f6-20020a5d6646000000b0033e34982311sm6475371wrw.81.2024.03.05.01.42.34
 for <win-pv-devel@lists.xenproject.org>
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 05 Mar 2024 01:42:34 -0800 (PST)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b2302060-dad4-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1709631755; x=1710236555; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:organization:reply-to
         :references:to:content-language:subject:user-agent:mime-version:date
         :message-id:from:from:to:cc:subject:date:message-id:reply-to;
        bh=OMKyE4IurgjiXjEOHgsxa+a08tR4r4g6uSOyxNmVtZw=;
        b=A3FRHEXaEFxnfS2Bx7FkN045caN1veVeejmvAV73xxCE7tko5BlBJgRBRbQipiofBv
         1ruKv0y0gEiRG6HtG4nmSbD7JPfSLIlLLQGzX13ViKHKZoMP/TuTI6yujMUt0tj7qwAq
         o1UrXtCNMIDaCG+RqYFI3RpNq4jjk/eArbA8JD7iBCtq/JpWcBM2LnwPkOfO0yxfWoue
         R0y7SkC+v2R4AjQYCfUz6rJiY1cGCm65Xu9c6GgglyioWBgjUSxkEoTsAs+JZ2CwJDvb
         /7ABRNMlpObMXSHIj1I2CYZMWOPGZF5Rh4pUoxmkmTda7nHH1B87dAhX0KqmiO99TRqh
         eapw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709631755; x=1710236555;
        h=content-transfer-encoding:in-reply-to:organization:reply-to
         :references:to:content-language:subject:user-agent:mime-version:date
         :message-id:from:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=OMKyE4IurgjiXjEOHgsxa+a08tR4r4g6uSOyxNmVtZw=;
        b=U3enrrjkJwf1djkP1OpIv4zH/l6QYERhARKCeAlTaZ4uHaASonoDAP8A3LaN/xzV2n
         PsLNIp2yDrYf/a/T3lcTrarxnYocHY1y8+atecU5HlAmsecBLqSOukCxQpdHLeoCoKuB
         y9l1I38HUL53eKx0NJJ0preatKWFjV87obfWFtcbETQIfTPD1oOlnV4uDsC0iYHTJNQl
         68gFAMH2GoEfBfgmpU550Jf+J5XxVFtqW0KhtDm2Z4B5iJItB9tGggCRslgswe6i2Zbl
         LHOBNLXZkR1hAtBUjVfn2FBct+NmW3yByP1C/0NE9OXSrleK8+g9JGwdpqARfkWLAQ44
         kZjQ==
X-Gm-Message-State: AOJu0YwpvAEqI2/TITFDMNfAK7Ljuf0kxjeEzFy5JpVSVhqAtsWtCo0F
	t+RmCd9TM89UHXiAgmwlPF9ZrZtKzlOrjNb2fTqlgndWOIyqu2kT5fmQJyI1eTo=
X-Google-Smtp-Source: AGHT+IH0CORc+8QeiOED3Yn6PIWNg5nB2N3s5DK9WL92hW+Cdf0R1Kd3WBtKZMDTU3KG+iaHlKHg6A==
X-Received: by 2002:a5d:480e:0:b0:33d:679d:a033 with SMTP id l14-20020a5d480e000000b0033d679da033mr9333022wrq.36.1709631754759;
        Tue, 05 Mar 2024 01:42:34 -0800 (PST)
From: Paul Durrant <xadimgnik@gmail.com>
X-Google-Original-From: Paul Durrant <paul@xen.org>
Message-ID: <590bb31a-488b-42ed-9b1a-4289c9e9948a@xen.org>
Date: Tue, 5 Mar 2024 09:42:32 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Adjust Packet's MDL when stripping ETHERNET_TAG from
 headers
Content-Language: en-US
To: win-pv-devel@lists.xenproject.org
References: <20240304071129.2-1-owen.smith@cloud.com>
Reply-To: paul@xen.org
Organization: Xen Project
In-Reply-To: <20240304071129.2-1-owen.smith@cloud.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 04/03/2024 07:11, Owen Smith wrote:
> Tagged headers have the ETHERNET_TAG stripped to convert to
> ETHERNET_UNTAGGED_HEADERs. As part of the fix up, the offsets for each
> header was adjusted, and the underlying MDL's MappedSystemVa was adjusted.
> However, the ByteCount and ByteOffset were not adjusted, resulting in
> instances of data corruption when MappedSystemVa != StartVa + ByteOffset.
> 
> It is possible to configure the default gateway to respond to ARP packets
> with TPID wrappings. Openvswitch will not strip this header since this commit:
> 
>      commit f0fb825a3785320430686834741c718ff4f8ebf4
>      Author: Eric Garver <e@erig.me>
>      Date:   Wed Mar 1 17:47:59 2017 -0500
> 
>          Add support for 802.1ad (QinQ tunneling)
> 
> Which means Windows was receiving corrupted ARP responses, leading to an
> inability to access the network correctly. This would usually show as
> Windows indicating that no internet access was available, or being unable
> it access other VMs or other resources beyond the default gateway.
> 
> Signed-off-by: Owen Smith <owen.smith@cloud.com>
> ---
>   src/xenvif/receiver.c | 13 ++++++++-----
>   1 file changed, 8 insertions(+), 5 deletions(-)
> 
> diff --git a/src/xenvif/receiver.c b/src/xenvif/receiver.c
> index 0ce5b67..720d1d1 100644
> --- a/src/xenvif/receiver.c
> +++ b/src/xenvif/receiver.c
> @@ -436,16 +436,23 @@ ReceiverRingProcessTag(
>   
>       Packet->TagControlInformation = NTOHS(EthernetHeader->Tagged.Tag.ControlInformation);
>   
> +    // Convert ETHERNET_TAGGED_HEADER into ETHERNET_UNTAGGED_HEADER
>       Offset = FIELD_OFFSET(ETHERNET_TAGGED_HEADER, Tag);
>       RtlMoveMemory((PUCHAR)EthernetHeader + sizeof (ETHERNET_TAG),
>                     (PUCHAR)EthernetHeader,
>                     Offset);
>   
>       // Fix up the packet information
> +    BaseVa = Packet->Mdl.MappedSystemVa;
> +    ASSERT(BaseVa != NULL);
> +
>       BaseVa += sizeof (ETHERNET_TAG);
>   
> -    BaseVa -= Packet->Offset;
>       Packet->Mdl.MappedSystemVa = BaseVa;

You only need...

> +    Packet->Mdl.ByteOffset += sizeof (ETHERNET_TAG);
> +    Packet->Mdl.ByteCount -= sizeof (ETHERNET_TAG);
> +
> +    ASSERT3P((PUCHAR)Packet->Mdl.StartVa + Packet->Mdl.ByteOffset, ==, Packet->Mdl.MappedSystemVa);
>   

these lines to achieve the aim of the patch AFAICT. Why mess with BaseVa?

   Paul

>       Packet->Length -= sizeof (ETHERNET_TAG);
>   
> @@ -468,10 +475,6 @@ ReceiverRingProcessTag(
>   
>       Info->Length -= sizeof (ETHERNET_TAG);
>   
> -    BaseVa += Packet->Offset;
> -
> -    EthernetHeader = (PETHERNET_HEADER)(BaseVa + Info->EthernetHeader.Offset);
> -
>       ASSERT3U(PayloadLength, ==, Packet->Length - Info->Length);
>   }
>   



From win-pv-devel-bounces@lists.xenproject.org Tue Mar 05 09:56:38 2024
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Tue, 05 Mar 2024 09:56:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.688674.1073127 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rhRXI-0007xY-RP; Tue, 05 Mar 2024 09:56:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 688674.1073127; Tue, 05 Mar 2024 09:56:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rhRXI-0007xR-Oq; Tue, 05 Mar 2024 09:56:36 +0000
Received: by outflank-mailman (input) for mailman id 688674;
 Tue, 05 Mar 2024 09:56:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=r+cG=KL=cloud.com=owen.smith@srs-se1.protection.inumbo.net>)
 id 1rhRXH-0007xL-Gw
 for win-pv-devel@lists.xenproject.org; Tue, 05 Mar 2024 09:56:35 +0000
Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com
 [2a00:1450:4864:20::12e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a5b547b6-dad6-11ee-afda-a90da7624cb6;
 Tue, 05 Mar 2024 10:56:34 +0100 (CET)
Received: by mail-lf1-x12e.google.com with SMTP id
 2adb3069b0e04-5133bd7eb47so3507767e87.3
 for <win-pv-devel@lists.xenproject.org>; Tue, 05 Mar 2024 01:56:33 -0800 (PST)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a5b547b6-dad6-11ee-afda-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709632593; x=1710237393; darn=lists.xenproject.org;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:from:to:cc:subject:date:message-id:reply-to;
        bh=uYhbYq4VrYsM69RsadRWP6HVPekEWevUiFJ8YZRYZqU=;
        b=ZndoPDjydAn/hbICGbYqOJf/jb6Cgj+v+vMDdSjuPkrZdc71PCWdjX6CENEQ0DucOe
         Ez0L8gUju80yImMQvbcU9O/89nnlltFS4GIJtIrsECGqay5TCE4EJd2uC7ddS4BDnj14
         Dve0UpS2LeZ3GNK+2BEnd2FW9jbbBqfp4ebsk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709632593; x=1710237393;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=uYhbYq4VrYsM69RsadRWP6HVPekEWevUiFJ8YZRYZqU=;
        b=cv5m2WsWcMPgoETKfl6jr9FCmJXPPU/07HT9SXfVk+eG/9z6OEKugeMlANU9ON0kfH
         ZCvb5VDFSG5dIlxfhqOuV33sqlOjhPlWnl1guPvvW5KP+EN1b71dKyT76RINuyUDyNnJ
         GLq8TGkX8Hw5neSP0yFVFakzIzyADA3uLipzK6GnO7yzXib8kQf7Dr+l+vj/QXrWBkB8
         h//ERhoUsYPim+MJHOOktpP4xMSWBi8+gBmPtqX0QmlM71/xHZV1/l8fRxCmVlpJIqL2
         Huz0f3PSCWGiqhO1CT6LE0KCqAELZofTYyBNmBtE9GfBH4FrmuX3n7PhD/iww/4s6zQ6
         5oHA==
X-Gm-Message-State: AOJu0YxqnJA0IuOOqR1dQEiLzFGm32LYLzwHoG23DqAlDLrT1OB+uiiP
	zVuPi3sBfF233j9hTMSQsdvvpvYlRgaK5z46aHEFnIvHTYQYtPdp9NzkGB/3YMBoD2WLvkwV62h
	eKA7FdIZbHZ8vATckJNQ2I4lRSGiy4+jynk6NrzRS+fikhbz1yjbG
X-Google-Smtp-Source: AGHT+IGlvpRQ0LQyDUNJiVTHOBMd2lNRj4LxBbQX3juJNQ/u6DtkojEElEOYmEgQg+Kj6LGM+ClEWXnWjnEdphLanvc=
X-Received: by 2002:a19:641a:0:b0:513:522a:a3b0 with SMTP id
 y26-20020a19641a000000b00513522aa3b0mr693155lfb.40.1709632592782; Tue, 05 Mar
 2024 01:56:32 -0800 (PST)
MIME-Version: 1.0
References: <20240304071129.2-1-owen.smith@cloud.com> <590bb31a-488b-42ed-9b1a-4289c9e9948a@xen.org>
In-Reply-To: <590bb31a-488b-42ed-9b1a-4289c9e9948a@xen.org>
From: Owen Smith <owen.smith@citrix.com>
Date: Tue, 5 Mar 2024 09:56:16 +0000
Message-ID: <CAC_UY8_uYwyk4Py+CTefEaNC8WuY_0AnqWxPkhmYAmusbv_upw@mail.gmail.com>
Subject: Re: [PATCH] Adjust Packet's MDL when stripping ETHERNET_TAG from headers
To: paul@xen.org
Cc: win-pv-devel@lists.xenproject.org
Content-Type: multipart/alternative; boundary="00000000000063ca270612e6db2d"

--00000000000063ca270612e6db2d
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

The addition of the TAG size then subtraction of the offset was confusing -
setting the BaseVa to MappedSystemVa then adjusting it clarified the intent
to adjust the MappedSystemVa (a cleaner approach would have been to adjust
the Mdl->MappedSystemVa +=3D sizeof(ETHERNET_TAG); but this would require
significant casting to function correctly).

> +    Packet->Mdl.ByteOffset +=3D sizeof (ETHERNET_TAG);
> +    Packet->Mdl.ByteCount -=3D sizeof (ETHERNET_TAG);

is the only required change.
Recalculating EthernetHeader is not required, as its not used again in that
function.

Owen


On Tue, Mar 5, 2024 at 9:43=E2=80=AFAM Paul Durrant <xadimgnik@gmail.com> w=
rote:

> On 04/03/2024 07:11, Owen Smith wrote:
> > Tagged headers have the ETHERNET_TAG stripped to convert to
> > ETHERNET_UNTAGGED_HEADERs. As part of the fix up, the offsets for each
> > header was adjusted, and the underlying MDL's MappedSystemVa was
> adjusted.
> > However, the ByteCount and ByteOffset were not adjusted, resulting in
> > instances of data corruption when MappedSystemVa !=3D StartVa + ByteOff=
set.
> >
> > It is possible to configure the default gateway to respond to ARP packe=
ts
> > with TPID wrappings. Openvswitch will not strip this header since this
> commit:
> >
> >      commit f0fb825a3785320430686834741c718ff4f8ebf4
> >      Author: Eric Garver <e@erig.me>
> >      Date:   Wed Mar 1 17:47:59 2017 -0500
> >
> >          Add support for 802.1ad (QinQ tunneling)
> >
> > Which means Windows was receiving corrupted ARP responses, leading to a=
n
> > inability to access the network correctly. This would usually show as
> > Windows indicating that no internet access was available, or being unab=
le
> > it access other VMs or other resources beyond the default gateway.
> >
> > Signed-off-by: Owen Smith <owen.smith@cloud.com>
> > ---
> >   src/xenvif/receiver.c | 13 ++++++++-----
> >   1 file changed, 8 insertions(+), 5 deletions(-)
> >
> > diff --git a/src/xenvif/receiver.c b/src/xenvif/receiver.c
> > index 0ce5b67..720d1d1 100644
> > --- a/src/xenvif/receiver.c
> > +++ b/src/xenvif/receiver.c
> > @@ -436,16 +436,23 @@ ReceiverRingProcessTag(
> >
> >       Packet->TagControlInformation =3D
> NTOHS(EthernetHeader->Tagged.Tag.ControlInformation);
> >
> > +    // Convert ETHERNET_TAGGED_HEADER into ETHERNET_UNTAGGED_HEADER
> >       Offset =3D FIELD_OFFSET(ETHERNET_TAGGED_HEADER, Tag);
> >       RtlMoveMemory((PUCHAR)EthernetHeader + sizeof (ETHERNET_TAG),
> >                     (PUCHAR)EthernetHeader,
> >                     Offset);
> >
> >       // Fix up the packet information
> > +    BaseVa =3D Packet->Mdl.MappedSystemVa;
> > +    ASSERT(BaseVa !=3D NULL);
> > +
> >       BaseVa +=3D sizeof (ETHERNET_TAG);
> >
> > -    BaseVa -=3D Packet->Offset;
> >       Packet->Mdl.MappedSystemVa =3D BaseVa;
>
> You only need...
>
> > +    Packet->Mdl.ByteOffset +=3D sizeof (ETHERNET_TAG);
> > +    Packet->Mdl.ByteCount -=3D sizeof (ETHERNET_TAG);
> > +
> > +    ASSERT3P((PUCHAR)Packet->Mdl.StartVa + Packet->Mdl.ByteOffset, =3D=
=3D,
> Packet->Mdl.MappedSystemVa);
> >
>
> these lines to achieve the aim of the patch AFAICT. Why mess with BaseVa?
>
>    Paul
>
> >       Packet->Length -=3D sizeof (ETHERNET_TAG);
> >
> > @@ -468,10 +475,6 @@ ReceiverRingProcessTag(
> >
> >       Info->Length -=3D sizeof (ETHERNET_TAG);
> >
> > -    BaseVa +=3D Packet->Offset;
> > -
> > -    EthernetHeader =3D (PETHERNET_HEADER)(BaseVa +
> Info->EthernetHeader.Offset);
> > -
> >       ASSERT3U(PayloadLength, =3D=3D, Packet->Length - Info->Length);
> >   }
> >
>
>
>

--00000000000063ca270612e6db2d
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_default" style=3D"font-family:monospac=
e">The addition of the TAG size then subtraction of the offset was confusin=
g - setting the BaseVa to MappedSystemVa then adjusting it clarified the in=
tent to adjust the MappedSystemVa (a cleaner approach would have been to ad=
just the Mdl-&gt;MappedSystemVa=C2=A0+=3D sizeof(ETHERNET_TAG); but this wo=
uld require significant casting to function correctly).</div><div class=3D"=
gmail_default" style=3D"font-family:monospace"><br></div><div class=3D"gmai=
l_default" style=3D"font-family:monospace"><span style=3D"color:rgb(80,0,80=
);font-family:Arial,Helvetica,sans-serif">&gt; +=C2=A0 =C2=A0 Packet-&gt;Md=
l.ByteOffset +=3D sizeof (ETHERNET_TAG);</span><br style=3D"color:rgb(80,0,=
80);font-family:Arial,Helvetica,sans-serif"><span style=3D"color:rgb(80,0,8=
0);font-family:Arial,Helvetica,sans-serif">&gt; +=C2=A0 =C2=A0 Packet-&gt;M=
dl.ByteCount -=3D sizeof (ETHERNET_TAG);</span><br></div><div class=3D"gmai=
l_default" style=3D"font-family:monospace"><span style=3D"color:rgb(80,0,80=
);font-family:Arial,Helvetica,sans-serif"><br></span></div><div class=3D"gm=
ail_default" style=3D"font-family:monospace"><span style=3D"color:rgb(80,0,=
80);font-family:Arial,Helvetica,sans-serif">is the only required change.</s=
pan></div><div class=3D"gmail_default" style=3D"font-family:monospace"><spa=
n style=3D"color:rgb(80,0,80);font-family:Arial,Helvetica,sans-serif">Recal=
culating=C2=A0</span><span style=3D"font-family:Arial,Helvetica,sans-serif"=
>EthernetHeader</span><span style=3D"font-family:Arial,Helvetica,sans-serif=
">=C2=A0is not required, as its not used again in that function.</span></di=
v><div class=3D"gmail_default" style=3D"font-family:monospace"><span style=
=3D"color:rgb(80,0,80);font-family:Arial,Helvetica,sans-serif"><br></span><=
/div><div class=3D"gmail_default" style=3D"font-family:monospace">Owen</div=
><div class=3D"gmail_default" style=3D"font-family:monospace"><br></div></d=
iv><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On =
Tue, Mar 5, 2024 at 9:43=E2=80=AFAM Paul Durrant &lt;<a href=3D"mailto:xadi=
mgnik@gmail.com">xadimgnik@gmail.com</a>&gt; wrote:<br></div><blockquote cl=
ass=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid=
 rgb(204,204,204);padding-left:1ex">On 04/03/2024 07:11, Owen Smith wrote:<=
br>
&gt; Tagged headers have the ETHERNET_TAG stripped to convert to<br>
&gt; ETHERNET_UNTAGGED_HEADERs. As part of the fix up, the offsets for each=
<br>
&gt; header was adjusted, and the underlying MDL&#39;s MappedSystemVa was a=
djusted.<br>
&gt; However, the ByteCount and ByteOffset were not adjusted, resulting in<=
br>
&gt; instances of data corruption when MappedSystemVa !=3D StartVa + ByteOf=
fset.<br>
&gt; <br>
&gt; It is possible to configure the default gateway to respond to ARP pack=
ets<br>
&gt; with TPID wrappings. Openvswitch will not strip this header since this=
 commit:<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0 commit f0fb825a3785320430686834741c718ff4f8ebf4<br=
>
&gt;=C2=A0 =C2=A0 =C2=A0 Author: Eric Garver &lt;<a href=3D"mailto:e@erig.m=
e" target=3D"_blank">e@erig.me</a>&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 Date:=C2=A0 =C2=A0Wed Mar 1 17:47:59 2017 -0500<br=
>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Add support for 802.1ad (QinQ tunnel=
ing)<br>
&gt; <br>
&gt; Which means Windows was receiving corrupted ARP responses, leading to =
an<br>
&gt; inability to access the network correctly. This would usually show as<=
br>
&gt; Windows indicating that no internet access was available, or being una=
ble<br>
&gt; it access other VMs or other resources beyond the default gateway.<br>
&gt; <br>
&gt; Signed-off-by: Owen Smith &lt;<a href=3D"mailto:owen.smith@cloud.com" =
target=3D"_blank">owen.smith@cloud.com</a>&gt;<br>
&gt; ---<br>
&gt;=C2=A0 =C2=A0src/xenvif/receiver.c | 13 ++++++++-----<br>
&gt;=C2=A0 =C2=A01 file changed, 8 insertions(+), 5 deletions(-)<br>
&gt; <br>
&gt; diff --git a/src/xenvif/receiver.c b/src/xenvif/receiver.c<br>
&gt; index 0ce5b67..720d1d1 100644<br>
&gt; --- a/src/xenvif/receiver.c<br>
&gt; +++ b/src/xenvif/receiver.c<br>
&gt; @@ -436,16 +436,23 @@ ReceiverRingProcessTag(<br>
&gt;=C2=A0 =C2=A0<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0Packet-&gt;TagControlInformation =3D NTOHS(E=
thernetHeader-&gt;Tagged.Tag.ControlInformation);<br>
&gt;=C2=A0 =C2=A0<br>
&gt; +=C2=A0 =C2=A0 // Convert ETHERNET_TAGGED_HEADER into ETHERNET_UNTAGGE=
D_HEADER<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0Offset =3D FIELD_OFFSET(ETHERNET_TAGGED_HEAD=
ER, Tag);<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0RtlMoveMemory((PUCHAR)EthernetHeader + sizeo=
f (ETHERNET_TAG),<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0(PUCHAR)EthernetHeader,<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0Offset);<br>
&gt;=C2=A0 =C2=A0<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0// Fix up the packet information<br>
&gt; +=C2=A0 =C2=A0 BaseVa =3D Packet-&gt;Mdl.MappedSystemVa;<br>
&gt; +=C2=A0 =C2=A0 ASSERT(BaseVa !=3D NULL);<br>
&gt; +<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0BaseVa +=3D sizeof (ETHERNET_TAG);<br>
&gt;=C2=A0 =C2=A0<br>
&gt; -=C2=A0 =C2=A0 BaseVa -=3D Packet-&gt;Offset;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0Packet-&gt;Mdl.MappedSystemVa =3D BaseVa;<br=
>
<br>
You only need...<br>
<br>
&gt; +=C2=A0 =C2=A0 Packet-&gt;Mdl.ByteOffset +=3D sizeof (ETHERNET_TAG);<b=
r>
&gt; +=C2=A0 =C2=A0 Packet-&gt;Mdl.ByteCount -=3D sizeof (ETHERNET_TAG);<br=
>
&gt; +<br>
&gt; +=C2=A0 =C2=A0 ASSERT3P((PUCHAR)Packet-&gt;Mdl.StartVa + Packet-&gt;Md=
l.ByteOffset, =3D=3D, Packet-&gt;Mdl.MappedSystemVa);<br>
&gt;=C2=A0 =C2=A0<br>
<br>
these lines to achieve the aim of the patch AFAICT. Why mess with BaseVa?<b=
r>
<br>
=C2=A0 =C2=A0Paul<br>
<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0Packet-&gt;Length -=3D sizeof (ETHERNET_TAG)=
;<br>
&gt;=C2=A0 =C2=A0<br>
&gt; @@ -468,10 +475,6 @@ ReceiverRingProcessTag(<br>
&gt;=C2=A0 =C2=A0<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0Info-&gt;Length -=3D sizeof (ETHERNET_TAG);<=
br>
&gt;=C2=A0 =C2=A0<br>
&gt; -=C2=A0 =C2=A0 BaseVa +=3D Packet-&gt;Offset;<br>
&gt; -<br>
&gt; -=C2=A0 =C2=A0 EthernetHeader =3D (PETHERNET_HEADER)(BaseVa + Info-&gt=
;EthernetHeader.Offset);<br>
&gt; -<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0ASSERT3U(PayloadLength, =3D=3D, Packet-&gt;L=
ength - Info-&gt;Length);<br>
&gt;=C2=A0 =C2=A0}<br>
&gt;=C2=A0 =C2=A0<br>
<br>
<br>
</blockquote></div>

--00000000000063ca270612e6db2d--


From win-pv-devel-bounces@lists.xenproject.org Tue Mar 05 09:58:13 2024
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Tue, 05 Mar 2024 09:58:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.688685.1073142 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rhRYr-000066-6j; Tue, 05 Mar 2024 09:58:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 688685.1073142; Tue, 05 Mar 2024 09:58:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rhRYr-00005z-44; Tue, 05 Mar 2024 09:58:13 +0000
Received: by outflank-mailman (input) for mailman id 688685;
 Tue, 05 Mar 2024 09:58:12 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=r+cG=KL=cloud.com=owen.smith@srs-se1.protection.inumbo.net>)
 id 1rhRYq-00005o-1g
 for win-pv-devel@lists.xenproject.org; Tue, 05 Mar 2024 09:58:12 +0000
Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com
 [2a00:1450:4864:20::12a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id dff0217e-dad6-11ee-afda-a90da7624cb6;
 Tue, 05 Mar 2024 10:58:11 +0100 (CET)
Received: by mail-lf1-x12a.google.com with SMTP id
 2adb3069b0e04-5133bd7eb47so3509140e87.3
 for <win-pv-devel@lists.xenproject.org>; Tue, 05 Mar 2024 01:58:11 -0800 (PST)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dff0217e-dad6-11ee-afda-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709632691; x=1710237491; darn=lists.xenproject.org;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:from:to:cc:subject:date:message-id:reply-to;
        bh=yfP7D8f+YM0aL5G+EzFSFniWJrNmoF9oBA+nJtsQO9M=;
        b=cQLsN4QPBcG502Mo9xlP7EAbM1K7D48BQMEDuAPCehaPXhY16I6iv6Z7R7eVJsxxgU
         wyNoxtMcm33HoG2w4yJqWN05Vl0S0PRpdwK/VhTeP1l9SVhf8gVI5W2FkUHe6tXqnTNT
         NqqKRGHfcp8nFKcuiYXAUj/GD+7VGotnRodas=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709632691; x=1710237491;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=yfP7D8f+YM0aL5G+EzFSFniWJrNmoF9oBA+nJtsQO9M=;
        b=UrXAvHpH4qhUtRmZ8KmTeKMj1zozFZLSxAUPPb9ORSVcudD2I74Vx/ngddZLRXLN9G
         3pwAy8zq/iq2JHsPlnvQxkfivK5eQVGuLAn+p3d17xyDRWrwLNailGLz5q4kM9jzlYZ9
         lofnfroBOAE/Yvt2RA65H+QxBRh+uhZh9Z6yrFkEeDf5SMP/c87WK2M88h1mbjdE7LFJ
         KxQ6PQD476UdC3wnM7Fyisxrdi0g6fRQRtFHi5QUKe6d63Ss+2S6HsZ+LczncBAZCl2H
         vnFaLv/AzlcsILz6yRFwQxuciCM43gFj4RRmsfgcZq5mo5Gp9VTMbMLX0Ne/i/3YViw6
         xkCQ==
X-Gm-Message-State: AOJu0Yy7re3NwcFNdRmYIK6PDTCXigII6gFtPnQyl2uSVjNyf7pWqZIu
	5GidaW4BSSEP/PQWzEheBJiWPwkRtCorT5ZszYMrI1Ghu0fpyJlVaFK3cXZ3p+bgCozoColmraN
	UoV9EyFzPAzAeNidKXRI4YycojOXv2/eOxNdD
X-Google-Smtp-Source: AGHT+IEIui4EpP5lUi7ZCN0SK2rfb/gcH/3eSoJbUUNhKHyrDNKST7yHdn94D4sCsQDbsya6AEzArM89Zsjyc6zPFHQ=
X-Received: by 2002:a05:6512:282c:b0:513:2c50:9644 with SMTP id
 cf44-20020a056512282c00b005132c509644mr1209532lfb.12.1709632690576; Tue, 05
 Mar 2024 01:58:10 -0800 (PST)
MIME-Version: 1.0
References: <20240304071129.2-1-owen.smith@cloud.com> <590bb31a-488b-42ed-9b1a-4289c9e9948a@xen.org>
 <CAC_UY8_uYwyk4Py+CTefEaNC8WuY_0AnqWxPkhmYAmusbv_upw@mail.gmail.com>
In-Reply-To: <CAC_UY8_uYwyk4Py+CTefEaNC8WuY_0AnqWxPkhmYAmusbv_upw@mail.gmail.com>
From: Owen Smith <owen.smith@citrix.com>
Date: Tue, 5 Mar 2024 09:57:54 +0000
Message-ID: <CAC_UY89bLvnReDC-tNcqR8FkWnXAAn8-xPG6iN63dWzcA7fotg@mail.gmail.com>
Subject: Re: [PATCH] Adjust Packet's MDL when stripping ETHERNET_TAG from headers
To: paul@xen.org
Cc: win-pv-devel@lists.xenproject.org
Content-Type: multipart/alternative; boundary="000000000000380dd40612e6e149"

--000000000000380dd40612e6e149
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Packet->Mdl.MappedSystemVa =3D (PUCHAR)Packet->Mdl.MappedSystemVa +
sizeof(ETHERNET_TAG);

might have been a cleaner statement of what the MDL manipulation is
actually doing

Owen

On Tue, Mar 5, 2024 at 9:56=E2=80=AFAM Owen Smith <owen.smith@citrix.com> w=
rote:

> The addition of the TAG size then subtraction of the offset was confusing
> - setting the BaseVa to MappedSystemVa then adjusting it clarified the
> intent to adjust the MappedSystemVa (a cleaner approach would have been t=
o
> adjust the Mdl->MappedSystemVa +=3D sizeof(ETHERNET_TAG); but this would
> require significant casting to function correctly).
>
> > +    Packet->Mdl.ByteOffset +=3D sizeof (ETHERNET_TAG);
> > +    Packet->Mdl.ByteCount -=3D sizeof (ETHERNET_TAG);
>
> is the only required change.
> Recalculating EthernetHeader is not required, as its not used again in
> that function.
>
> Owen
>
>
> On Tue, Mar 5, 2024 at 9:43=E2=80=AFAM Paul Durrant <xadimgnik@gmail.com>=
 wrote:
>
>> On 04/03/2024 07:11, Owen Smith wrote:
>> > Tagged headers have the ETHERNET_TAG stripped to convert to
>> > ETHERNET_UNTAGGED_HEADERs. As part of the fix up, the offsets for each
>> > header was adjusted, and the underlying MDL's MappedSystemVa was
>> adjusted.
>> > However, the ByteCount and ByteOffset were not adjusted, resulting in
>> > instances of data corruption when MappedSystemVa !=3D StartVa +
>> ByteOffset.
>> >
>> > It is possible to configure the default gateway to respond to ARP
>> packets
>> > with TPID wrappings. Openvswitch will not strip this header since this
>> commit:
>> >
>> >      commit f0fb825a3785320430686834741c718ff4f8ebf4
>> >      Author: Eric Garver <e@erig.me>
>> >      Date:   Wed Mar 1 17:47:59 2017 -0500
>> >
>> >          Add support for 802.1ad (QinQ tunneling)
>> >
>> > Which means Windows was receiving corrupted ARP responses, leading to =
an
>> > inability to access the network correctly. This would usually show as
>> > Windows indicating that no internet access was available, or being
>> unable
>> > it access other VMs or other resources beyond the default gateway.
>> >
>> > Signed-off-by: Owen Smith <owen.smith@cloud.com>
>> > ---
>> >   src/xenvif/receiver.c | 13 ++++++++-----
>> >   1 file changed, 8 insertions(+), 5 deletions(-)
>> >
>> > diff --git a/src/xenvif/receiver.c b/src/xenvif/receiver.c
>> > index 0ce5b67..720d1d1 100644
>> > --- a/src/xenvif/receiver.c
>> > +++ b/src/xenvif/receiver.c
>> > @@ -436,16 +436,23 @@ ReceiverRingProcessTag(
>> >
>> >       Packet->TagControlInformation =3D
>> NTOHS(EthernetHeader->Tagged.Tag.ControlInformation);
>> >
>> > +    // Convert ETHERNET_TAGGED_HEADER into ETHERNET_UNTAGGED_HEADER
>> >       Offset =3D FIELD_OFFSET(ETHERNET_TAGGED_HEADER, Tag);
>> >       RtlMoveMemory((PUCHAR)EthernetHeader + sizeof (ETHERNET_TAG),
>> >                     (PUCHAR)EthernetHeader,
>> >                     Offset);
>> >
>> >       // Fix up the packet information
>> > +    BaseVa =3D Packet->Mdl.MappedSystemVa;
>> > +    ASSERT(BaseVa !=3D NULL);
>> > +
>> >       BaseVa +=3D sizeof (ETHERNET_TAG);
>> >
>> > -    BaseVa -=3D Packet->Offset;
>> >       Packet->Mdl.MappedSystemVa =3D BaseVa;
>>
>> You only need...
>>
>> > +    Packet->Mdl.ByteOffset +=3D sizeof (ETHERNET_TAG);
>> > +    Packet->Mdl.ByteCount -=3D sizeof (ETHERNET_TAG);
>> > +
>> > +    ASSERT3P((PUCHAR)Packet->Mdl.StartVa + Packet->Mdl.ByteOffset, =
=3D=3D,
>> Packet->Mdl.MappedSystemVa);
>> >
>>
>> these lines to achieve the aim of the patch AFAICT. Why mess with BaseVa=
?
>>
>>    Paul
>>
>> >       Packet->Length -=3D sizeof (ETHERNET_TAG);
>> >
>> > @@ -468,10 +475,6 @@ ReceiverRingProcessTag(
>> >
>> >       Info->Length -=3D sizeof (ETHERNET_TAG);
>> >
>> > -    BaseVa +=3D Packet->Offset;
>> > -
>> > -    EthernetHeader =3D (PETHERNET_HEADER)(BaseVa +
>> Info->EthernetHeader.Offset);
>> > -
>> >       ASSERT3U(PayloadLength, =3D=3D, Packet->Length - Info->Length);
>> >   }
>> >
>>
>>
>>

--000000000000380dd40612e6e149
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_default" style=3D"font-family:monospac=
e"><span style=3D"font-family:Arial,Helvetica,sans-serif">Packet-&gt;Mdl.Ma=
ppedSystemVa =3D (PUCHAR)</span><span style=3D"font-family:Arial,Helvetica,=
sans-serif">Packet-&gt;Mdl.MappedSystemVa + sizeof(ETHERNET_TAG)</span><spa=
n style=3D"font-family:Arial,Helvetica,sans-serif">;</span><br></div><div c=
lass=3D"gmail_default" style=3D"font-family:monospace"><span style=3D"font-=
family:Arial,Helvetica,sans-serif"><br></span></div><div class=3D"gmail_def=
ault" style=3D"font-family:monospace"><span style=3D"font-family:Arial,Helv=
etica,sans-serif">might have been a cleaner statement of what the MDL manip=
ulation is actually doing</span></div><div class=3D"gmail_default" style=3D=
"font-family:monospace"><span style=3D"font-family:Arial,Helvetica,sans-ser=
if"><br></span></div><div class=3D"gmail_default" style=3D"font-family:mono=
space"><span style=3D"font-family:Arial,Helvetica,sans-serif">Owen</span></=
div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_at=
tr">On Tue, Mar 5, 2024 at 9:56=E2=80=AFAM Owen Smith &lt;<a href=3D"mailto=
:owen.smith@citrix.com">owen.smith@citrix.com</a>&gt; wrote:<br></div><bloc=
kquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:=
1px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr"><div class=3D=
"gmail_default" style=3D"font-family:monospace">The addition of the TAG siz=
e then subtraction of the offset was confusing - setting the BaseVa to Mapp=
edSystemVa then adjusting it clarified the intent to adjust the MappedSyste=
mVa (a cleaner approach would have been to adjust the Mdl-&gt;MappedSystemV=
a=C2=A0+=3D sizeof(ETHERNET_TAG); but this would require significant castin=
g to function correctly).</div><div class=3D"gmail_default" style=3D"font-f=
amily:monospace"><br></div><div class=3D"gmail_default" style=3D"font-famil=
y:monospace"><span style=3D"color:rgb(80,0,80);font-family:Arial,Helvetica,=
sans-serif">&gt; +=C2=A0 =C2=A0 Packet-&gt;Mdl.ByteOffset +=3D sizeof (ETHE=
RNET_TAG);</span><br style=3D"color:rgb(80,0,80);font-family:Arial,Helvetic=
a,sans-serif"><span style=3D"color:rgb(80,0,80);font-family:Arial,Helvetica=
,sans-serif">&gt; +=C2=A0 =C2=A0 Packet-&gt;Mdl.ByteCount -=3D sizeof (ETHE=
RNET_TAG);</span><br></div><div class=3D"gmail_default" style=3D"font-famil=
y:monospace"><span style=3D"color:rgb(80,0,80);font-family:Arial,Helvetica,=
sans-serif"><br></span></div><div class=3D"gmail_default" style=3D"font-fam=
ily:monospace"><span style=3D"color:rgb(80,0,80);font-family:Arial,Helvetic=
a,sans-serif">is the only required change.</span></div><div class=3D"gmail_=
default" style=3D"font-family:monospace"><span style=3D"color:rgb(80,0,80);=
font-family:Arial,Helvetica,sans-serif">Recalculating=C2=A0</span><span sty=
le=3D"font-family:Arial,Helvetica,sans-serif">EthernetHeader</span><span st=
yle=3D"font-family:Arial,Helvetica,sans-serif">=C2=A0is not required, as it=
s not used again in that function.</span></div><div class=3D"gmail_default"=
 style=3D"font-family:monospace"><span style=3D"color:rgb(80,0,80);font-fam=
ily:Arial,Helvetica,sans-serif"><br></span></div><div class=3D"gmail_defaul=
t" style=3D"font-family:monospace">Owen</div><div class=3D"gmail_default" s=
tyle=3D"font-family:monospace"><br></div></div><br><div class=3D"gmail_quot=
e"><div dir=3D"ltr" class=3D"gmail_attr">On Tue, Mar 5, 2024 at 9:43=E2=80=
=AFAM Paul Durrant &lt;<a href=3D"mailto:xadimgnik@gmail.com" target=3D"_bl=
ank">xadimgnik@gmail.com</a>&gt; wrote:<br></div><blockquote class=3D"gmail=
_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204=
,204);padding-left:1ex">On 04/03/2024 07:11, Owen Smith wrote:<br>
&gt; Tagged headers have the ETHERNET_TAG stripped to convert to<br>
&gt; ETHERNET_UNTAGGED_HEADERs. As part of the fix up, the offsets for each=
<br>
&gt; header was adjusted, and the underlying MDL&#39;s MappedSystemVa was a=
djusted.<br>
&gt; However, the ByteCount and ByteOffset were not adjusted, resulting in<=
br>
&gt; instances of data corruption when MappedSystemVa !=3D StartVa + ByteOf=
fset.<br>
&gt; <br>
&gt; It is possible to configure the default gateway to respond to ARP pack=
ets<br>
&gt; with TPID wrappings. Openvswitch will not strip this header since this=
 commit:<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0 commit f0fb825a3785320430686834741c718ff4f8ebf4<br=
>
&gt;=C2=A0 =C2=A0 =C2=A0 Author: Eric Garver &lt;<a href=3D"mailto:e@erig.m=
e" target=3D"_blank">e@erig.me</a>&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 Date:=C2=A0 =C2=A0Wed Mar 1 17:47:59 2017 -0500<br=
>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Add support for 802.1ad (QinQ tunnel=
ing)<br>
&gt; <br>
&gt; Which means Windows was receiving corrupted ARP responses, leading to =
an<br>
&gt; inability to access the network correctly. This would usually show as<=
br>
&gt; Windows indicating that no internet access was available, or being una=
ble<br>
&gt; it access other VMs or other resources beyond the default gateway.<br>
&gt; <br>
&gt; Signed-off-by: Owen Smith &lt;<a href=3D"mailto:owen.smith@cloud.com" =
target=3D"_blank">owen.smith@cloud.com</a>&gt;<br>
&gt; ---<br>
&gt;=C2=A0 =C2=A0src/xenvif/receiver.c | 13 ++++++++-----<br>
&gt;=C2=A0 =C2=A01 file changed, 8 insertions(+), 5 deletions(-)<br>
&gt; <br>
&gt; diff --git a/src/xenvif/receiver.c b/src/xenvif/receiver.c<br>
&gt; index 0ce5b67..720d1d1 100644<br>
&gt; --- a/src/xenvif/receiver.c<br>
&gt; +++ b/src/xenvif/receiver.c<br>
&gt; @@ -436,16 +436,23 @@ ReceiverRingProcessTag(<br>
&gt;=C2=A0 =C2=A0<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0Packet-&gt;TagControlInformation =3D NTOHS(E=
thernetHeader-&gt;Tagged.Tag.ControlInformation);<br>
&gt;=C2=A0 =C2=A0<br>
&gt; +=C2=A0 =C2=A0 // Convert ETHERNET_TAGGED_HEADER into ETHERNET_UNTAGGE=
D_HEADER<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0Offset =3D FIELD_OFFSET(ETHERNET_TAGGED_HEAD=
ER, Tag);<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0RtlMoveMemory((PUCHAR)EthernetHeader + sizeo=
f (ETHERNET_TAG),<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0(PUCHAR)EthernetHeader,<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0Offset);<br>
&gt;=C2=A0 =C2=A0<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0// Fix up the packet information<br>
&gt; +=C2=A0 =C2=A0 BaseVa =3D Packet-&gt;Mdl.MappedSystemVa;<br>
&gt; +=C2=A0 =C2=A0 ASSERT(BaseVa !=3D NULL);<br>
&gt; +<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0BaseVa +=3D sizeof (ETHERNET_TAG);<br>
&gt;=C2=A0 =C2=A0<br>
&gt; -=C2=A0 =C2=A0 BaseVa -=3D Packet-&gt;Offset;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0Packet-&gt;Mdl.MappedSystemVa =3D BaseVa;<br=
>
<br>
You only need...<br>
<br>
&gt; +=C2=A0 =C2=A0 Packet-&gt;Mdl.ByteOffset +=3D sizeof (ETHERNET_TAG);<b=
r>
&gt; +=C2=A0 =C2=A0 Packet-&gt;Mdl.ByteCount -=3D sizeof (ETHERNET_TAG);<br=
>
&gt; +<br>
&gt; +=C2=A0 =C2=A0 ASSERT3P((PUCHAR)Packet-&gt;Mdl.StartVa + Packet-&gt;Md=
l.ByteOffset, =3D=3D, Packet-&gt;Mdl.MappedSystemVa);<br>
&gt;=C2=A0 =C2=A0<br>
<br>
these lines to achieve the aim of the patch AFAICT. Why mess with BaseVa?<b=
r>
<br>
=C2=A0 =C2=A0Paul<br>
<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0Packet-&gt;Length -=3D sizeof (ETHERNET_TAG)=
;<br>
&gt;=C2=A0 =C2=A0<br>
&gt; @@ -468,10 +475,6 @@ ReceiverRingProcessTag(<br>
&gt;=C2=A0 =C2=A0<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0Info-&gt;Length -=3D sizeof (ETHERNET_TAG);<=
br>
&gt;=C2=A0 =C2=A0<br>
&gt; -=C2=A0 =C2=A0 BaseVa +=3D Packet-&gt;Offset;<br>
&gt; -<br>
&gt; -=C2=A0 =C2=A0 EthernetHeader =3D (PETHERNET_HEADER)(BaseVa + Info-&gt=
;EthernetHeader.Offset);<br>
&gt; -<br>
&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0ASSERT3U(PayloadLength, =3D=3D, Packet-&gt;L=
ength - Info-&gt;Length);<br>
&gt;=C2=A0 =C2=A0}<br>
&gt;=C2=A0 =C2=A0<br>
<br>
<br>
</blockquote></div>
</blockquote></div>

--000000000000380dd40612e6e149--


From win-pv-devel-bounces@lists.xenproject.org Tue Mar 05 10:24:00 2024
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Tue, 05 Mar 2024 10:24:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.688704.1073175 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rhRxn-0006IF-Ki; Tue, 05 Mar 2024 10:23:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 688704.1073175; Tue, 05 Mar 2024 10:23:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rhRxn-0006I8-ID; Tue, 05 Mar 2024 10:23:59 +0000
Received: by outflank-mailman (input) for mailman id 688704;
 Tue, 05 Mar 2024 10:23:58 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=HlLy=KL=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1rhRxm-0005r6-JR
 for win-pv-devel@lists.xenproject.org; Tue, 05 Mar 2024 10:23:58 +0000
Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com
 [2a00:1450:4864:20::32f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 79fb103a-dada-11ee-afda-a90da7624cb6;
 Tue, 05 Mar 2024 11:23:58 +0100 (CET)
Received: by mail-wm1-x32f.google.com with SMTP id
 5b1f17b1804b1-412eced6d1aso3082055e9.3
 for <win-pv-devel@lists.xenproject.org>; Tue, 05 Mar 2024 02:23:58 -0800 (PST)
Received: from [192.168.19.5] (54-240-197-238.amazon.com. [54.240.197.238])
 by smtp.gmail.com with ESMTPSA id
 l27-20020a05600c1d1b00b00412c7b91ef0sm14113621wms.37.2024.03.05.02.23.56
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 05 Mar 2024 02:23:56 -0800 (PST)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 79fb103a-dada-11ee-afda-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1709634237; x=1710239037; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:organization:content-language
         :references:cc:to:subject:reply-to:user-agent:mime-version:date
         :message-id:from:from:to:cc:subject:date:message-id:reply-to;
        bh=Rd26bBPRfDXKOHKu8pMez2seAVtdxc7pWDfz2/Ntfj8=;
        b=fMiJJ9hwGC3bXwUY+oLKMf6fBjC5Ewt8vS6XdR9hLIqK7EJgzc0pKVQx6BVPBU3FHR
         HIsS+Oz42PCFwog56nIUHAbjmSqBXmGy7AjPXiQt0e9RmvKsCzqnHiMm+nD0Py3C9zNv
         dumzQWl8Yw5TY89ulTcLv/w80k2Zx8SRfRbFRNSPqPHnIe1wgdUfMSGfvRatj50+mxJ8
         eUJkB0dvNbSiRju2RQtWOM+qtrsa5CjtT4OvHShud8YAfRGtmMmAjANr/esbKcJIzAvL
         5jTZjHhlrIx5ZcuICt/XDjLTBvFntA2+n72HCV7Mvyfzfh0BBZ/5/j1ozKwFdVw7Matd
         z1OQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709634237; x=1710239037;
        h=content-transfer-encoding:in-reply-to:organization:content-language
         :references:cc:to:subject:reply-to:user-agent:mime-version:date
         :message-id:from:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Rd26bBPRfDXKOHKu8pMez2seAVtdxc7pWDfz2/Ntfj8=;
        b=WB6cln/UFqJWtZcYyrX2+64xfiy0iLqB+cvK0ucx7KUtPbGgBYjFA08xMOuQ9xU1nq
         O4E8KPWGOjUhfKRzS3yncKeUHfYkHhgD9hVRpjlPRDTDl0JeDwfz015GWGHsWYQcG0CX
         MQ9j9+uEogOFF7uKQp9pvx5in273vj2evTybJV7Wc2w7amGCb1vdoCN/r5TD61xIfhrj
         VKP1FlR1qxD9Kx2DX9TXKdjfhAT7EMce1BZ/6N8EdBD0jekU7DsWUEHO88EFmmF1Sg+t
         sDtiWf25JF0vlLNCwWpxhkoqNOSrBzt+v8VrruOrkd1jHvugvXmGU6MTm9qF1KHW4mBr
         qZew==
X-Gm-Message-State: AOJu0Yxn/Wh582SUgaorxOhQoIaf1G7p1u5ehuY9EXUSOuxyS3tDFtJa
	O+OaZ3LVE3A6HYasYRN4SzLq0muqg6AtF1aRyyLUfVaSjvRL8gun
X-Google-Smtp-Source: AGHT+IHEeVc76pLHnTPrYsNDgPNackqXqg/zc/ht0AIbTFoyxQLaxcHPT7vS1Q/mSDUCFWulP7rkHA==
X-Received: by 2002:a05:600c:1384:b0:412:c213:d0fc with SMTP id u4-20020a05600c138400b00412c213d0fcmr9436777wmf.15.1709634237250;
        Tue, 05 Mar 2024 02:23:57 -0800 (PST)
From: Paul Durrant <xadimgnik@gmail.com>
X-Google-Original-From: Paul Durrant <paul@xen.org>
Message-ID: <adecb342-f878-487e-8bb6-2679a9616a71@xen.org>
Date: Tue, 5 Mar 2024 10:23:54 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Reply-To: paul@xen.org
Subject: Re: [PATCH] Adjust Packet's MDL when stripping ETHERNET_TAG from
 headers
To: Owen Smith <owen.smith@citrix.com>
Cc: win-pv-devel@lists.xenproject.org
References: <20240304071129.2-1-owen.smith@cloud.com>
 <590bb31a-488b-42ed-9b1a-4289c9e9948a@xen.org>
 <CAC_UY8_uYwyk4Py+CTefEaNC8WuY_0AnqWxPkhmYAmusbv_upw@mail.gmail.com>
 <CAC_UY89bLvnReDC-tNcqR8FkWnXAAn8-xPG6iN63dWzcA7fotg@mail.gmail.com>
Content-Language: en-US
Organization: Xen Project
In-Reply-To: <CAC_UY89bLvnReDC-tNcqR8FkWnXAAn8-xPG6iN63dWzcA7fotg@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 05/03/2024 09:57, Owen Smith wrote:
> Packet->Mdl.MappedSystemVa = (PUCHAR)Packet->Mdl.MappedSystemVa + 
> sizeof(ETHERNET_TAG);
> 
> might have been a cleaner statement of what the MDL manipulation is 
> actually doing
> 

Yes, I prefer that. Just want a patch that is really doing what it says 
without making other tangential changes.

   Paul



From win-pv-devel-bounces@lists.xenproject.org Tue Mar 05 11:03:11 2024
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Tue, 05 Mar 2024 11:03:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.688727.1073210 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rhSZg-0005OA-4r; Tue, 05 Mar 2024 11:03:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 688727.1073210; Tue, 05 Mar 2024 11:03:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rhSZg-0005O3-28; Tue, 05 Mar 2024 11:03:08 +0000
Received: by outflank-mailman (input) for mailman id 688727;
 Tue, 05 Mar 2024 11:03:07 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=r+cG=KL=cloud.com=owen.smith@srs-se1.protection.inumbo.net>)
 id 1rhSZf-0005Nx-17
 for win-pv-devel@lists.xenproject.org; Tue, 05 Mar 2024 11:03:07 +0000
Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com
 [2a00:1450:4864:20::12f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f0e5234b-dadf-11ee-a1ee-f123f15fe8a2;
 Tue, 05 Mar 2024 12:03:05 +0100 (CET)
Received: by mail-lf1-x12f.google.com with SMTP id
 2adb3069b0e04-51336ab1fb7so4054277e87.1
 for <win-pv-devel@lists.xenproject.org>; Tue, 05 Mar 2024 03:03:05 -0800 (PST)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f0e5234b-dadf-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709636584; x=1710241384; darn=lists.xenproject.org;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:from:to:cc:subject:date:message-id:reply-to;
        bh=R8sJEsLfvyu6TxB+h2jb2XAZ8nCBpG/M4hgbsSBYCU8=;
        b=n2ABWu0opx5G0gp/xJ+zx85AY1WM70wixiznOc1UOVfKNeKplSvcMI06Mc3Zvj8hgo
         w12cARFj3bFs+dwzIchkroId+7B2R1ZHqiFnRLMp2ow2exlug35bGdQI7YTxjQqj7gkQ
         PS6x8YrIoG4eDOBjq+CMA/A59IJVnJEvD88eE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709636584; x=1710241384;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=R8sJEsLfvyu6TxB+h2jb2XAZ8nCBpG/M4hgbsSBYCU8=;
        b=E1naqhsUwT87Q5Ul2HTexNnE5jWjArINGLv3gtib35qgs2un0ttIQxwDR+/hfVNJU4
         iV4ah6qwUrDu5T49KJ7NH+xOzV0F9UtScnJUXZzXtH2rIj+xGQjOma1XCmi9F4uqQuqY
         FO7bn2y/qsBEh1M/VD+/tZA8jyxFioD//1Q57EtP570GEmHLFbolbQNtYQT4qkfFaMuw
         qETpIhnDkGAXcYqu07Ui8ETMZn9PhSNJFYGaQQ2ToiCEgvSS7HpLe89B2yV56r09gbBn
         Mw+0yQNwy2VDXdUebWvM4QzgRgWcAEFcu1LOJxyYvxAsBqXnmiXByaY1aCcTUdkMs3mp
         01VQ==
X-Gm-Message-State: AOJu0Yy7fBuKy9lXqTimoenLmKII6B2aVTnPt6pNYJIESdgBAVLgcNQj
	g/KiBbNtUSX4fN0YxgT8f12sOBgSPlHHW2jRChNuuomGMrHVzgbmNyxiK4HqQ1w49fZk6H1shdH
	TkgEm2WAJAjBUWsNHwo1pWxnOmRB7pNwU9iGqFxyJFFyh2pPVinS1
X-Google-Smtp-Source: AGHT+IFeXKgEMv97l96WvcYI8/WWIkWMl/PPkGimMU3JWAHmIVCuLKrmauFUNHyltmjcZ1UDiQiGHMQGiDe2QUDJsVo=
X-Received: by 2002:a05:6512:1246:b0:513:40f7:dcaf with SMTP id
 fb6-20020a056512124600b0051340f7dcafmr1270291lfb.8.1709636584353; Tue, 05 Mar
 2024 03:03:04 -0800 (PST)
MIME-Version: 1.0
References: <20240304071129.2-1-owen.smith@cloud.com> <590bb31a-488b-42ed-9b1a-4289c9e9948a@xen.org>
 <CAC_UY8_uYwyk4Py+CTefEaNC8WuY_0AnqWxPkhmYAmusbv_upw@mail.gmail.com>
 <CAC_UY89bLvnReDC-tNcqR8FkWnXAAn8-xPG6iN63dWzcA7fotg@mail.gmail.com> <adecb342-f878-487e-8bb6-2679a9616a71@xen.org>
In-Reply-To: <adecb342-f878-487e-8bb6-2679a9616a71@xen.org>
From: Owen Smith <owen.smith@citrix.com>
Date: Tue, 5 Mar 2024 11:02:48 +0000
Message-ID: <CAC_UY89+9xuxit49P9ZviXgoNr0XvTUNwA9xAEipi=01XtvOTA@mail.gmail.com>
Subject: Re: [PATCH] Adjust Packet's MDL when stripping ETHERNET_TAG from headers
To: paul@xen.org
Cc: win-pv-devel@lists.xenproject.org
Content-Type: multipart/alternative; boundary="0000000000004e602a0612e7c9e8"

--0000000000004e602a0612e7c9e8
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

I'll resend with the changes.
I'll also split the removal of the EthernetHeader recalculation into a
separate patch.

Owen

On Tue, Mar 5, 2024 at 10:23=E2=80=AFAM Paul Durrant <xadimgnik@gmail.com> =
wrote:

> On 05/03/2024 09:57, Owen Smith wrote:
> > Packet->Mdl.MappedSystemVa =3D (PUCHAR)Packet->Mdl.MappedSystemVa +
> > sizeof(ETHERNET_TAG);
> >
> > might have been a cleaner statement of what the MDL manipulation is
> > actually doing
> >
>
> Yes, I prefer that. Just want a patch that is really doing what it says
> without making other tangential changes.
>
>    Paul
>
>

--0000000000004e602a0612e7c9e8
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_default" style=3D"font-family:monospac=
e">I&#39;ll resend with the changes.</div><div class=3D"gmail_default" styl=
e=3D"font-family:monospace">I&#39;ll also split the removal of the Ethernet=
Header recalculation into a separate=C2=A0patch.</div><div class=3D"gmail_d=
efault" style=3D"font-family:monospace"><br></div><div class=3D"gmail_defau=
lt" style=3D"font-family:monospace">Owen</div></div><br><div class=3D"gmail=
_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Tue, Mar 5, 2024 at 10:23=
=E2=80=AFAM Paul Durrant &lt;<a href=3D"mailto:xadimgnik@gmail.com">xadimgn=
ik@gmail.com</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" styl=
e=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);paddin=
g-left:1ex">On 05/03/2024 09:57, Owen Smith wrote:<br>
&gt; Packet-&gt;Mdl.MappedSystemVa =3D (PUCHAR)Packet-&gt;Mdl.MappedSystemV=
a + <br>
&gt; sizeof(ETHERNET_TAG);<br>
&gt; <br>
&gt; might have been a cleaner statement of what the MDL manipulation is <b=
r>
&gt; actually doing<br>
&gt; <br>
<br>
Yes, I prefer that. Just want a patch that is really doing what it says <br=
>
without making other tangential changes.<br>
<br>
=C2=A0 =C2=A0Paul<br>
<br>
</blockquote></div>

--0000000000004e602a0612e7c9e8--


From win-pv-devel-bounces@lists.xenproject.org Tue Mar 05 15:33:53 2024
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Tue, 05 Mar 2024 15:33:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.688832.1073513 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rhWng-00075k-2L; Tue, 05 Mar 2024 15:33:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 688832.1073513; Tue, 05 Mar 2024 15:33:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rhWnf-00075d-W4; Tue, 05 Mar 2024 15:33:51 +0000
Received: by outflank-mailman (input) for mailman id 688832;
 Tue, 05 Mar 2024 15:33:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=r+cG=KL=cloud.com=owen.smith@srs-se1.protection.inumbo.net>)
 id 1rhWne-00073r-EY
 for win-pv-devel@lists.xenproject.org; Tue, 05 Mar 2024 15:33:50 +0000
Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com
 [2a00:1450:4864:20::42c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c321d46a-db05-11ee-afda-a90da7624cb6;
 Tue, 05 Mar 2024 16:33:49 +0100 (CET)
Received: by mail-wr1-x42c.google.com with SMTP id
 ffacd0b85a97d-33dcad9e3a2so3169970f8f.3
 for <win-pv-devel@lists.xenproject.org>; Tue, 05 Mar 2024 07:33:49 -0800 (PST)
Received: from localhost.localdomain
 (host86-169-131-93.range86-169.btcentralplus.com. [86.169.131.93])
 by smtp.gmail.com with ESMTPSA id
 b7-20020a05600003c700b0033e44b23921sm3943875wrg.24.2024.03.05.07.33.46
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 05 Mar 2024 07:33:47 -0800 (PST)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c321d46a-db05-11ee-afda-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1709652827; x=1710257627; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=RJ+RcwzdVCnKONac/2puUJ5aP1irlYfn6j3EuFoMg1U=;
        b=MrI7ReCDKwTs+qhS2Dw7d7/wnFRYNamM+PcsfhWwXH5esvQVbW42ck2hvEmkYUkM3R
         sDWYHjFmou954T+ibtgJYPor5ftF1YAwd13kWGIxuNlyLLje7v82CfjpQaSNom6nwDLZ
         NL4tK2IZe3UrFYm7SmoUb8mCxfMxn0n9L+tVo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709652827; x=1710257627;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=RJ+RcwzdVCnKONac/2puUJ5aP1irlYfn6j3EuFoMg1U=;
        b=P74Pk6lIqPTPci/zFWAn8alHRAQM2SrrLvSybUQ2g2gPbo5YZxSz3JaLvQaOc3mlDB
         sI+ydfHoL1VHG9bIRR4Qb78sfT2Np+LqsZu7yXg3R1uhc676jxkqV/lVlsZzl3238uj1
         EK7OSPXZ/VLdxdB8BgmFhjffqSqHpht+gFugZ5zcb0zn3eC7ZgsG57vqcOpnZ20e3DRV
         D4eI8nakos99z7w6XOL0v6QK52AHwhUQNuEwQYEnlsu8pk8dmLy5xnKZBhbv8c5bUWth
         Jn7hJA+S+GKWZS+4hlD9iE92iQJ99lRzFaTeqVVukCI+hi2qnVsxjpLgbVy2QsIDChMS
         HZVg==
X-Gm-Message-State: AOJu0YxadjH5elzoXAht/WWSe0aZg8Rs/2xMjo+a+kt3GRQaEYRG7amr
	TFEp9UuogyvgfK2TEDNxa3/KDDJUhfMwpQgy6Uoz9pJmBHkA6Z1XkcT9SSR1SO3Sa1Nfu2LDKGe
	MSA==
X-Google-Smtp-Source: AGHT+IGYDUAi6Lgg3GEQsT+JYf0BtfAFv+y+6/njbCjkwrj+wSJmwimsbiSRMsL3zt73yqNNLGk25w==
X-Received: by 2002:adf:db42:0:b0:33d:e267:fe2e with SMTP id f2-20020adfdb42000000b0033de267fe2emr7831828wrj.59.1709652827355;
        Tue, 05 Mar 2024 07:33:47 -0800 (PST)
From: Owen Smith <owen.smith@cloud.com>
To: win-pv-devel@lists.xenproject.org
Cc: Owen Smith <owen.smith@cloud.com>
Subject: [PATCH 2/2] Remove unnecessary assignment of EthernetHeader
Date: Tue,  5 Mar 2024 15:33:31 +0000
Message-ID: <20240305153331.1555-2-owen.smith@cloud.com>
X-Mailer: git-send-email 2.41.0.windows.3
In-Reply-To: <20240305153331.1555-1-owen.smith@cloud.com>
References: <20240305153331.1555-1-owen.smith@cloud.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

BaseVa and EthernetHeader are not used after this point in the function,
remove the unnecessary assignment

Signed-off-by: Owen Smith <owen.smith@cloud.com>
---
 src/xenvif/receiver.c | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/src/xenvif/receiver.c b/src/xenvif/receiver.c
index 37e2c60..df5ed6f 100644
--- a/src/xenvif/receiver.c
+++ b/src/xenvif/receiver.c
@@ -469,10 +469,6 @@ ReceiverRingProcessTag(
 
     Info->Length -= sizeof (ETHERNET_TAG);
 
-    BaseVa += Packet->Offset;
-
-    EthernetHeader = (PETHERNET_HEADER)(BaseVa + Info->EthernetHeader.Offset);
-
     ASSERT3U(PayloadLength, ==, Packet->Length - Info->Length);
 }
 
-- 
2.41.0.windows.3



From win-pv-devel-bounces@lists.xenproject.org Tue Mar 05 15:33:53 2024
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Tue, 05 Mar 2024 15:33:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.688831.1073509 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rhWnf-00074C-0t; Tue, 05 Mar 2024 15:33:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 688831.1073509; Tue, 05 Mar 2024 15:33:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rhWne-000745-Ub; Tue, 05 Mar 2024 15:33:50 +0000
Received: by outflank-mailman (input) for mailman id 688831;
 Tue, 05 Mar 2024 15:33:49 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=r+cG=KL=cloud.com=owen.smith@srs-se1.protection.inumbo.net>)
 id 1rhWnd-00073r-QC
 for win-pv-devel@lists.xenproject.org; Tue, 05 Mar 2024 15:33:49 +0000
Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com
 [2a00:1450:4864:20::42b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c2a66bf4-db05-11ee-afda-a90da7624cb6;
 Tue, 05 Mar 2024 16:33:48 +0100 (CET)
Received: by mail-wr1-x42b.google.com with SMTP id
 ffacd0b85a97d-33e285a33bdso2149755f8f.2
 for <win-pv-devel@lists.xenproject.org>; Tue, 05 Mar 2024 07:33:48 -0800 (PST)
Received: from localhost.localdomain
 (host86-169-131-93.range86-169.btcentralplus.com. [86.169.131.93])
 by smtp.gmail.com with ESMTPSA id
 b7-20020a05600003c700b0033e44b23921sm3943875wrg.24.2024.03.05.07.33.46
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 05 Mar 2024 07:33:46 -0800 (PST)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c2a66bf4-db05-11ee-afda-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1709652827; x=1710257627; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=veFze7pgMi6wSVA8hPKWpj8Z17+T4ldc7ZysdvwoEYI=;
        b=KnXqzHS9otyqSaDiBW2+TgdXzIMir4iyFCKUqhPA/8Avz+MbpnY47IwQO3hRVdRwWD
         QAnkoF1WuLEjxw7Me2V/bklEawrvvX3dFQ/OrLWN+XffLzmWWkL7FOFREY54J5UB+D7e
         XaAjJDLcINOmwYnl4OnRikvzp0Wq8xl9+VSfk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709652827; x=1710257627;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=veFze7pgMi6wSVA8hPKWpj8Z17+T4ldc7ZysdvwoEYI=;
        b=aX0zjE9a3eZ9t6pxFwgMb3Ucr+NTepxcTJTXSZGMTwzpmr33BHQDuhiQW5WenI9XRB
         Y2JpvbuBAMy6Lu5UKtIZk1kD4Ngmd1KvHpr80qaR3yxFYq7z16e8JwYEu0dxgYNnP7Y8
         vWV9Iokj6vVQGbeAkXWReVUbjgl3ewWxxgAGQPYoRKCzyUD4286Fwgt9eGlvd3dPf0cf
         WcY36n+PSLxAINT5gZv3NA8Nk7viE4YPWGzAYVPd+vys3TI4dIYP2msxNTDFQralgpuN
         iEooPeQG0fTLAbQcnyALJQCtltKycwV0DRomhYq3ZykMKzl0OX8kDCED7JIle7K6Gf46
         8hPQ==
X-Gm-Message-State: AOJu0YzFkmpRyA+klPCRsruJT92k96YlKwlQJwAAEOE5J+rCG2yapCL1
	PlVjJ41DW8kndjePmhPwRAd3wHux+qCXt9JJvIgwuzdwpkfQicj2FMevLQc7vHMrfydE+SwSyDA
	hXg==
X-Google-Smtp-Source: AGHT+IFnqztwDcBHnfTjN9OOPdHuneh0V7q9h8r2hwqaqk90u2lfZRVs7PtULPMPBpW/jZJESN++Ow==
X-Received: by 2002:a5d:66cf:0:b0:33e:10b7:a1ba with SMTP id k15-20020a5d66cf000000b0033e10b7a1bamr8780480wrw.29.1709652826747;
        Tue, 05 Mar 2024 07:33:46 -0800 (PST)
From: Owen Smith <owen.smith@cloud.com>
To: win-pv-devel@lists.xenproject.org
Cc: Owen Smith <owen.smith@cloud.com>
Subject: [PATCH 1/2] Adjust Packet's MDL when stripping ETHERNET_TAG from headers
Date: Tue,  5 Mar 2024 15:33:30 +0000
Message-ID: <20240305153331.1555-1-owen.smith@cloud.com>
X-Mailer: git-send-email 2.41.0.windows.3
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Tagged headers have the ETHERNET_TAG stripped to convert to
ETHERNET_UNTAGGED_HEADERs. As part of the fix up, the offsets for each
header was adjusted, and the underlying MDL's MappedSystemVa was adjusted.
However, the ByteCount and ByteOffset were not adjusted, resulting in
instances of data corruption when MappedSystemVa != StartVa + ByteOffset.

It is possible to configure the default gateway to respond to ARP packets
with TPID wrappings. Openvswitch will not strip this header since this commit:

    commit f0fb825a3785320430686834741c718ff4f8ebf4
    Author: Eric Garver <e@erig.me>
    Date:   Wed Mar 1 17:47:59 2017 -0500

        Add support for 802.1ad (QinQ tunneling)

Which means Windows was receiving corrupted ARP responses, leading to an
inability to access the network correctly. This would usually show as
Windows indicating that no internet access was available, or being unable
it access other VMs or other resources beyond the default gateway.

Signed-off-by: Owen Smith <owen.smith@cloud.com>
---
 src/xenvif/receiver.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/src/xenvif/receiver.c b/src/xenvif/receiver.c
index 0ce5b67..37e2c60 100644
--- a/src/xenvif/receiver.c
+++ b/src/xenvif/receiver.c
@@ -442,10 +442,11 @@ ReceiverRingProcessTag(
                   Offset);
 
     // Fix up the packet information
-    BaseVa += sizeof (ETHERNET_TAG);
+    Packet->Mdl.MappedSystemVa = (PUCHAR)Packet->Mdl.MappedSystemVa + sizeof(ETHERNET_TAG);
+    Packet->Mdl.ByteOffset += sizeof(ETHERNET_TAG);
+    Packet->Mdl.ByteCount -= sizeof(ETHERNET_TAG);
 
-    BaseVa -= Packet->Offset;
-    Packet->Mdl.MappedSystemVa = BaseVa;
+    ASSERT3P((PUCHAR)Packet->Mdl.StartVa + Packet->Mdl.ByteOffset, ==, Packet->Mdl.MappedSystemVa);
 
     Packet->Length -= sizeof (ETHERNET_TAG);
 
-- 
2.41.0.windows.3



From win-pv-devel-bounces@lists.xenproject.org Thu Mar 07 08:51:10 2024
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Thu, 07 Mar 2024 08:51:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.689683.1074912 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1ri9T2-0004L3-KU; Thu, 07 Mar 2024 08:51:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 689683.1074912; Thu, 07 Mar 2024 08:51:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1ri9T2-0004Kw-Hr; Thu, 07 Mar 2024 08:51:08 +0000
Received: by outflank-mailman (input) for mailman id 689683;
 Thu, 07 Mar 2024 08:51:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kUX/=KN=invisiblethingslab.com=omeg@srs-se1.protection.inumbo.net>)
 id 1ri9T0-0004Kq-AY
 for win-pv-devel@lists.xenproject.org; Thu, 07 Mar 2024 08:51:06 +0000
Received: from wout4-smtp.messagingengine.com (wout4-smtp.messagingengine.com
 [64.147.123.20]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d0bd8862-dc5f-11ee-a1ee-f123f15fe8a2;
 Thu, 07 Mar 2024 09:51:00 +0100 (CET)
Received: from compute4.internal (compute4.nyi.internal [10.202.2.44])
 by mailout.west.internal (Postfix) with ESMTP id 4EAE532009F8;
 Thu,  7 Mar 2024 03:50:56 -0500 (EST)
Received: from mailfrontend2 ([10.202.2.163])
 by compute4.internal (MEProxy); Thu, 07 Mar 2024 03:50:56 -0500
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu,
 7 Mar 2024 03:50:54 -0500 (EST)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d0bd8862-dc5f-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:message-id:mime-version:reply-to:subject:subject:to:to; s=fm1;
	 t=1709801455; x=1709887855; bh=vo4L4jeGWPqDZj/SmRtYixLwN4WDW9xk
	9LoBpBnlB78=; b=WBHsB0HFJJoc+vzY5C7a7N1ZL6zSCjLLgZ8dTtU6s/n+hNrf
	ZEy9NWKdje9x0mCi1VH0BEsObFKWyo3f4vAiLURC0jFrgbjkcEvbzR48hqF/rYdt
	nr+hh+0LMOLJe+tZ4kft/EQP5JHPqv3LsgCUfNYSrswPqymnd2xcUR2jnX8y6Kt5
	zUPsUD2J/Q4CRHv4+5sQ4HfJ2rcse++LeZKE/eM/+zPM2oQgcQ8oaBeAkMxh1Scc
	ht+jdTzTvLtqj4GhXVGkaPyeoeH9pk/h+je83xfiWYuLwvstSglnGLJdvnI9bzFI
	9tIIj2tMMLHyvnsgeba8Ujafubs9fMACExCfGw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:message-id:mime-version:reply-to:subject
	:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender
	:x-sasl-enc; s=fm1; t=1709801455; x=1709887855; bh=vo4L4jeGWPqDZ
	j/SmRtYixLwN4WDW9xk9LoBpBnlB78=; b=aNHHvWITelhj/zgjJtYYupFo7g8gZ
	9Bog+HwUnJCcBpgxUvqGQqoFcxvU09EWfngX7gCdjEoAPygyyLYEWq1nV4mwzQwt
	bvN4goEUAoeHNs9rB3wEjpZd/079MYqH3ZTTnvHlt30j90elEwvUP96O7uhh57u1
	wyR0p0QvS+Hd/xeyQbiGlTZjLO12BNmxgLCuA0GkXMzspAsaOIcZfP+M6NtfmdKd
	NgvJ9Sp3xX+KGd/OnJT7u6EyldFrYvCzftxSw6/KqRxeOLGHif0i4JVqHvuC4Vo1
	1klJk6iNZYrplHDpCq2f6KnV58byd7h+Elv6lA6zcPvW4EQ9AcjW1DGsA==
X-ME-Sender: <xms:73_pZcTYQqojqvqD8j_D_BUKkkGmNJuAqoX1_wNOPZzPXe2wH6wrdw>
    <xme:73_pZZx_3KEVQrLwevyot38vopRkzUzjnEWaiggcZC0TBObfympY5tA-6vo39Im7U
    FfA78PXe32An9U>
X-ME-Received: <xmr:73_pZZ3VuOipQan1eKT9jjBaSD9EFMcP8L3Wwq4MrfhMOfHh05YW7RvLIgxx31dqSsVXa32qnOgmqc0JwWX9N2miNX7BNrfboXPtfDL5LXg>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledriedvgdduvddvucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucenucfjughrpefhvfevufffkffogggtgfesthekre
    dtredtjeenucfhrhhomheptfgrfhgrlhcuhghojhguhihlrgcuoehomhgvghesihhnvhhi
    shhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpeffhfekge
    elgfevtddvgffgjeetfffgfefggeejueelhffgvdehgefggeeftdeutdenucevlhhushht
    vghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehomhgvghesihhnvhhish
    hisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:73_pZQBuPX4hbyq37GvIPDCEE5FWgAL3kGTcg3tIEzkSoggD8Ib3tQ>
    <xmx:73_pZViaP5Vp2XjRxpE4qH-RPgAqiuu0PlObER82Gt1v8vyG2W0c7w>
    <xmx:73_pZcpjxsqebMUSlSYjpQ936L5ZVz-XpYKoI8HYSeHFk5KZucQjgg>
    <xmx:73_pZfeJ_pWR4uJVkl4VzxxjHZ_mlccbbeX9SeB9lHkmfsAl0C4r5A>
Feedback-ID: i409c4082:Fastmail
From: =?UTF-8?q?Rafa=C5=82=20Wojdy=C5=82a?= <omeg@invisiblethingslab.com>
To: win-pv-devel@lists.xenproject.org
Cc: =?UTF-8?q?Rafa=C5=82=20Wojdy=C5=82a?= <omeg@invisiblethingslab.com>
Subject: [PATCH] Fix double-free on error condition in GnttabPermitForeignAccess()
Date: Thu,  7 Mar 2024 09:50:31 +0100
Message-ID: <20240307085031.1121-1-omeg@invisiblethingslab.com>
X-Mailer: git-send-email 2.43.0.windows.1
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

XENIFACE_GNTTAB_CONTEXT associated with the request was incorrectly freed
by GnttabPermitForeignAccess() when a failure occured. The context is also
freed by the parent function, IoctlGnttabPermitForeignAccess(), which led
to a double-free and kernel heap corruption.

Signed-off-by: Rafał Wojdyła <omeg@invisiblethingslab.com>
---
 src/xeniface/ioctl_gnttab.c | 7 -------
 1 file changed, 7 deletions(-)

diff --git a/src/xeniface/ioctl_gnttab.c b/src/xeniface/ioctl_gnttab.c
index 8ab2099..083c29d 100644
--- a/src/xeniface/ioctl_gnttab.c
+++ b/src/xeniface/ioctl_gnttab.c
@@ -303,13 +303,6 @@ fail2:
 
 fail1:
     Error("Fail1\n");
-    GnttabStopSharing(Fdo, Context, Page);
-
-    if (Context != NULL) {
-        RtlZeroMemory(Context, sizeof(*Context));
-        __FreePoolWithTag(Context, XENIFACE_POOL_TAG);
-    }
-
     return Status;
 }
 
-- 
2.43.0.windows.1



From win-pv-devel-bounces@lists.xenproject.org Thu Mar 07 09:36:47 2024
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Thu, 07 Mar 2024 09:36:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.689697.1074925 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1riABC-0002gF-0u; Thu, 07 Mar 2024 09:36:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 689697.1074925; Thu, 07 Mar 2024 09:36:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1riABB-0002g8-UY; Thu, 07 Mar 2024 09:36:45 +0000
Received: by outflank-mailman (input) for mailman id 689697;
 Thu, 07 Mar 2024 09:36:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WJta=KN=cloud.com=owen.smith@srs-se1.protection.inumbo.net>)
 id 1riABA-0002g2-SX
 for win-pv-devel@lists.xenproject.org; Thu, 07 Mar 2024 09:36:44 +0000
Received: from mail-lj1-x22e.google.com (mail-lj1-x22e.google.com
 [2a00:1450:4864:20::22e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 34dcfdb5-dc66-11ee-afda-a90da7624cb6;
 Thu, 07 Mar 2024 10:36:44 +0100 (CET)
Received: by mail-lj1-x22e.google.com with SMTP id
 38308e7fff4ca-2d269b2ff48so8403871fa.3
 for <win-pv-devel@lists.xenproject.org>; Thu, 07 Mar 2024 01:36:42 -0800 (PST)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 34dcfdb5-dc66-11ee-afda-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709804202; x=1710409002; darn=lists.xenproject.org;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:from:to:cc:subject:date:message-id:reply-to;
        bh=8HqBleNtTZ+Q/ypBt76c7/nebFYMwDsXsx85nsDUec4=;
        b=f2ddpemi4UFaHdqPbTh4DYEKJocchLmKibGr9CEKB0HJVQnq6OqQy2Bgehzm6GZpY0
         VMOTEvNFW+r0As5NzyHQwqU4nfAsLuDwyyK/+dbe1nH5oi8jlHo6Oj5c3BxLNAMX9M94
         q/ygucA/JB96ZcMv4OFxSVNfXB2CfHQ5QFLv0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709804202; x=1710409002;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=8HqBleNtTZ+Q/ypBt76c7/nebFYMwDsXsx85nsDUec4=;
        b=F3lmYSOe3v8ky/A9yW5k3i9OucwSr6Ufk0beUoca4DmlHNnYlPz5Mt/34Nz0Rg5huh
         zvgs4mHL5snMa8WCj5rni6bzCmnGTspQgRZz9ZGsx0G/EZF3T0BZx152baLN0aGtBBTg
         0Q1/ZElHWRAgnz3UnjzbTEjcXkrqWwMG99KadlwrdC89EM8DaqhKkOtBTe4n/jhZK62W
         pzsxCXBaga9A5VSeeI4tpP8lcPXglbmv0bEOc4wp2NBqVm7T+2sGGRmjREQdHrm6Wp3t
         D0CD4/Jex5nZulCbDfmhMHr+sashPm1btrE6PvAAgWWxCquclD/Olzk+Yss+/6NfqDOl
         +hSA==
X-Gm-Message-State: AOJu0YyybNaGnlOcFWAtTKr9CXO/rUfs6gAiT8v/l60sgO4VYmDtRGS1
	f2YvvxAkBSa791/EmK5/qQbZs4+0tm8J5LYwO2zpNhxYmr0dvo5APycnA3qx85wWiNSTT7okxtI
	IcWwwqnexT3P7bA+iWbfUKpOuryyj9tHNTMpy
X-Google-Smtp-Source: AGHT+IFdmTcHnzQ82/k75qDh4eY14p3LaFAmlBFkfM5ekFA6iNWyLEiTey/2BWTE/8pLEPdPkTVvG7RHUHIdcxsragg=
X-Received: by 2002:ac2:4ec5:0:b0:513:410c:d0e7 with SMTP id
 p5-20020ac24ec5000000b00513410cd0e7mr878325lfr.50.1709804202021; Thu, 07 Mar
 2024 01:36:42 -0800 (PST)
MIME-Version: 1.0
References: <20240307085031.1121-1-omeg@invisiblethingslab.com>
In-Reply-To: <20240307085031.1121-1-omeg@invisiblethingslab.com>
From: Owen Smith <owen.smith@citrix.com>
Date: Thu, 7 Mar 2024 09:36:29 +0000
Message-ID: <CAC_UY88wFwye8evg1sZq5j=hqTqmM92xLRMznNHHRUKWiwUPdA@mail.gmail.com>
Subject: Re: [PATCH] Fix double-free on error condition in GnttabPermitForeignAccess()
To: =?UTF-8?B?UmFmYcWCIFdvamR5xYJh?= <omeg@invisiblethingslab.com>
Cc: win-pv-devel@lists.xenproject.org
Content-Type: multipart/alternative; boundary="00000000000018fcd006130ed06a"

--00000000000018fcd006130ed06a
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Is there other cleanup needed in other fail cases?
Context->Grants, Context->KernelVa, Context->Mdl are allocated and pages
could need revoking access during the cleanup operation in a failure case.

Calling GnttabStopSharing() should do this, but its not called in the case
where GnttabPermitForeignAccess returns a failure code.

Owen

On Thu, Mar 7, 2024 at 8:51=E2=80=AFAM Rafa=C5=82 Wojdy=C5=82a <omeg@invisi=
blethingslab.com>
wrote:

> XENIFACE_GNTTAB_CONTEXT associated with the request was incorrectly freed
> by GnttabPermitForeignAccess() when a failure occured. The context is als=
o
> freed by the parent function, IoctlGnttabPermitForeignAccess(), which led
> to a double-free and kernel heap corruption.
>
> Signed-off-by: Rafa=C5=82 Wojdy=C5=82a <omeg@invisiblethingslab.com>
> ---
>  src/xeniface/ioctl_gnttab.c | 7 -------
>  1 file changed, 7 deletions(-)
>
> diff --git a/src/xeniface/ioctl_gnttab.c b/src/xeniface/ioctl_gnttab.c
> index 8ab2099..083c29d 100644
> --- a/src/xeniface/ioctl_gnttab.c
> +++ b/src/xeniface/ioctl_gnttab.c
> @@ -303,13 +303,6 @@ fail2:
>
>  fail1:
>      Error("Fail1\n");
> -    GnttabStopSharing(Fdo, Context, Page);
> -
> -    if (Context !=3D NULL) {
> -        RtlZeroMemory(Context, sizeof(*Context));
> -        __FreePoolWithTag(Context, XENIFACE_POOL_TAG);
> -    }
> -
>      return Status;
>  }
>
> --
> 2.43.0.windows.1
>
>
>

--00000000000018fcd006130ed06a
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_default" style=3D"font-family:monospac=
e">Is there other cleanup needed in other fail cases?</div><div class=3D"gm=
ail_default" style=3D"font-family:monospace">Context-&gt;Grants, Context-&g=
t;KernelVa, Context-&gt;Mdl are allocated and pages could need revoking acc=
ess during the cleanup operation in a failure case.</div><div class=3D"gmai=
l_default" style=3D"font-family:monospace"><br></div><div class=3D"gmail_de=
fault" style=3D"font-family:monospace">Calling=C2=A0GnttabStopSharing() sho=
uld do this, but its=C2=A0not called in the case where=C2=A0GnttabPermitFor=
eignAccess returns a failure code.</div><div class=3D"gmail_default" style=
=3D"font-family:monospace"><br></div><div class=3D"gmail_default" style=3D"=
font-family:monospace">Owen</div></div><br><div class=3D"gmail_quote"><div =
dir=3D"ltr" class=3D"gmail_attr">On Thu, Mar 7, 2024 at 8:51=E2=80=AFAM Raf=
a=C5=82 Wojdy=C5=82a &lt;<a href=3D"mailto:omeg@invisiblethingslab.com">ome=
g@invisiblethingslab.com</a>&gt; wrote:<br></div><blockquote class=3D"gmail=
_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204=
,204);padding-left:1ex">XENIFACE_GNTTAB_CONTEXT associated with the request=
 was incorrectly freed<br>
by GnttabPermitForeignAccess() when a failure occured. The context is also<=
br>
freed by the parent function, IoctlGnttabPermitForeignAccess(), which led<b=
r>
to a double-free and kernel heap corruption.<br>
<br>
Signed-off-by: Rafa=C5=82 Wojdy=C5=82a &lt;<a href=3D"mailto:omeg@invisible=
thingslab.com" target=3D"_blank">omeg@invisiblethingslab.com</a>&gt;<br>
---<br>
=C2=A0src/xeniface/ioctl_gnttab.c | 7 -------<br>
=C2=A01 file changed, 7 deletions(-)<br>
<br>
diff --git a/src/xeniface/ioctl_gnttab.c b/src/xeniface/ioctl_gnttab.c<br>
index 8ab2099..083c29d 100644<br>
--- a/src/xeniface/ioctl_gnttab.c<br>
+++ b/src/xeniface/ioctl_gnttab.c<br>
@@ -303,13 +303,6 @@ fail2:<br>
<br>
=C2=A0fail1:<br>
=C2=A0 =C2=A0 =C2=A0Error(&quot;Fail1\n&quot;);<br>
-=C2=A0 =C2=A0 GnttabStopSharing(Fdo, Context, Page);<br>
-<br>
-=C2=A0 =C2=A0 if (Context !=3D NULL) {<br>
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 RtlZeroMemory(Context, sizeof(*Context));<br>
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 __FreePoolWithTag(Context, XENIFACE_POOL_TAG);=
<br>
-=C2=A0 =C2=A0 }<br>
-<br>
=C2=A0 =C2=A0 =C2=A0return Status;<br>
=C2=A0}<br>
<br>
-- <br>
2.43.0.windows.1<br>
<br>
<br>
</blockquote></div>

--00000000000018fcd006130ed06a--


From win-pv-devel-bounces@lists.xenproject.org Thu Mar 07 10:07:07 2024
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Thu, 07 Mar 2024 10:07:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.689715.1074950 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1riAeX-0008G3-L9; Thu, 07 Mar 2024 10:07:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 689715.1074950; Thu, 07 Mar 2024 10:07:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1riAeX-0008Fv-IP; Thu, 07 Mar 2024 10:07:05 +0000
Received: by outflank-mailman (input) for mailman id 689715;
 Thu, 07 Mar 2024 10:07:04 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kUX/=KN=invisiblethingslab.com=omeg@srs-se1.protection.inumbo.net>)
 id 1riAeW-0008Fp-KE
 for win-pv-devel@lists.xenproject.org; Thu, 07 Mar 2024 10:07:04 +0000
Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com
 [64.147.123.19]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6f097abc-dc6a-11ee-a1ee-f123f15fe8a2;
 Thu, 07 Mar 2024 11:07:01 +0100 (CET)
Received: from compute3.internal (compute3.nyi.internal [10.202.2.43])
 by mailout.west.internal (Postfix) with ESMTP id C18843200488;
 Thu,  7 Mar 2024 05:06:56 -0500 (EST)
Received: from mailfrontend2 ([10.202.2.163])
 by compute3.internal (MEProxy); Thu, 07 Mar 2024 05:06:56 -0500
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu,
 7 Mar 2024 05:06:55 -0500 (EST)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6f097abc-dc6a-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-type:content-type:date
	:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to; s=fm1; t=1709806016;
	 x=1709892416; bh=BxvnG4G+H2lmmg/qWLAHmOwZaJ+hZITAiiADHLzMWpM=; b=
	jr0Hw1j1WxUVErnZCBftzozkw99xDxwkW/tLuv1N5zf1//bJFKR47A6s0/xg7ZMa
	hDk4OBoqZb8Vvj9HLlrgihfndhKcf0pWkXGyB0DLne2zx/njV1NOG2ht6IigiBXU
	SfMHuhMhbASZgObwwTJuXIK9iefDbvq7goeZLNAnqeIRzFVZJYDGa9Mf06/1vIoK
	p1Q71YdWJ9Tf+umtXIyOtsk325VuzYwx0JfKv8m5eZQtkHa9+aJXw2Cd5Z01CVw6
	iwnwSntY4Jkh89a5doigRw0n6A+SpTeYEPu9xKr7qY1djubvSOQ5D6o0WrLsV1fj
	hjXmFALcCde966aWCoqPMA==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-type:content-type:date:date
	:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=
	fm1; t=1709806016; x=1709892416; bh=BxvnG4G+H2lmmg/qWLAHmOwZaJ+h
	ZITAiiADHLzMWpM=; b=jVXtJxRu97amtqIAKM7jbg86RDjN6A3lO5TCe0o4SbLJ
	A0FVN+K2wOdKFKz+z0cuhRQ/TA8iWZsHE9iMrkhnTvOJ0JSAqFEOfdthnb8eY5Rb
	0EDWor6l3KVSVldstLVtZ3kv2sFomt1SpK4RSSoR6u0tGyD8sGTk0DFrxehZ1edt
	3U0yEF/6OTWAnSHaR116hQgzRE8AciLLHLepclOBfZjNZmXC11Y0Rx3917JfbTvj
	Jq/0vA0VQtyGyEl8EthIZhDywVe9mmcZI00QpXqhaUHADRlakWZjzDFE4UGjo4bk
	ZHGXvd1wlEbpA2EWs33SihHZZcbePdYUyX7VCJMlRQ==
X-ME-Sender: <xms:wJHpZZ46jMP1vKV-ROgpXG7dVH0WNUf29s3rRNT7fpebeVPGu1bTXA>
    <xme:wJHpZW4F_ZRe8IE2ZdZtP3CqFiNbehWCjzf5GMqzG_O_9rBlex4Gd__W5TBjZgCVW
    hE1GcZfTX1LfpI>
X-ME-Received: <xmr:wJHpZQd8ZB-gOHK8eMaHug1XjV2kASxCv73c84PcaO3RcUQuUBiNuDbSDTfftYz6T23BvsNZ6IyVhsZTeLXCHHdM0RGkACcAMYQ>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrieefgdduudcutefuodetggdotefrodftvf
    curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu
    uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc
    fjughrpefkffggfgfuvfevfhfhjggtsehgtderredtvdejnecuhfhrohhmpeftrghfrghl
    ucghohhjugihlhgruceoohhmvghgsehinhhvihhsihgslhgvthhhihhnghhslhgrsgdrtg
    homheqnecuggftrfgrthhtvghrnhepudeihfdtteehiedtgfeukefhudelteegheeivdfh
    keehtdffkedtveefueeuvdelnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpe
    hmrghilhhfrhhomhepohhmvghgsehinhhvihhsihgslhgvthhhihhnghhslhgrsgdrtgho
    mh
X-ME-Proxy: <xmx:wJHpZSKFjlLDY2z1u6phXfe6vLgLfPGroym-X4tTwW-XAU8v0Lcqdw>
    <xmx:wJHpZdL_5qVfxxXSBeRJtApUElKZ3jQOoch0GsYwvn-BipHj7YPWug>
    <xmx:wJHpZbw0sJ8VgrD01joU0l_hEfp8JTmdU7Hh2NG1Sc0hHN13bi0p5Q>
    <xmx:wJHpZQk4p-XSuNNY7fkwo-LZH3auUfNCAyFCwXH7NpAg7NV8y9trWw>
Feedback-ID: i409c4082:Fastmail
Message-ID: <bf6b6df0-aa58-44d0-9c59-9a5b022b5620@invisiblethingslab.com>
Date: Thu, 7 Mar 2024 11:06:52 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Fix double-free on error condition in
 GnttabPermitForeignAccess()
Content-Language: en-US
To: Owen Smith <owen.smith@citrix.com>
Cc: win-pv-devel@lists.xenproject.org
References: <20240307085031.1121-1-omeg@invisiblethingslab.com>
 <CAC_UY88wFwye8evg1sZq5j=hqTqmM92xLRMznNHHRUKWiwUPdA@mail.gmail.com>
From: =?UTF-8?B?UmFmYcWCIFdvamR5xYJh?= <omeg@invisiblethingslab.com>
Autocrypt: addr=omeg@invisiblethingslab.com;
 keydata= xsBNBFH+JQwBCAC0ym1Jtm5HM2p32Nw8NDNdkhwJR1For9txOKqeYMJWBTPzGJQG9HVHdYY/
 PZNfzvJkl26q1CB7JbAXVq2rSt1hn7cc4qL4BIDackJ4SEAAYbSLK82pQYUHhj18nNzZgxnn
 DBHpppRUA76DhSRKxEOZ+7GQAHd6H8RA0zBW+5ut0iOmglia3sOlim2yqeBRj6XaRn3RGmT9
 LXQu/UrJDY52LwJGE9on1wTvw+tN4QmCipFUk6YToVbkHiyDSnNN6aRqclH0vJBZquagQ/wn
 aOohowIyyzbY2+GJspKEPD3J9Ov5aKe/jN13WjBYwcy+NUG9SWT+VTIi6th43mh/L3dNABEB
 AAHNLVJhZmHFgiBXb2pkecWCYSA8b21lZ0BpbnZpc2libGV0aGluZ3NsYWIuY29tPsLAeQQT
 AQIAIwUCUkHE5wIbAwcLCQgHAwIBBhUIAgkKCwQWAgMBAh4BAheAAAoJEIWi9rB2GrW7hasI
 AK5DFMRDqARqlOIncq0bDJbbPnScXozcVjrU2jHaA736d9eoXNixSxX+D1+4QGjgEIUJWaa6
 yqGTlqgHdOMKWYoDpUsm56Utj16E/ln1hSxXxEeHL30Fyqfqfmi0mlJv4ijasfxqwoHSt+Yy
 cACIhm563cwCaAdaO++lrcF2O/g+KIUXo0uEx+n6c8M25gu2TC3Cxn1ZeWH+5y9rUB6ph6JV
 eVKAYBSdK4qDEpDwz124YcpBloh5S4Jt6pVBSbJodbP+LQyet3Gb3IMYzFa+M6SMO/6EQX8k
 Q0zv1axlFCFs/Wf19A+nh40fZRRL+Q/v3lY9/v7u2mY4jGvKYao+sKXOwE0EUf4lDAEIALD5
 ungRVF1VnOKpNBS6xcpXw7jGX5Lh+r9fOZ9pSyqQRfD5t6yF3bjwgJBzfIJO2t/WhxlNTwZA
 Y3ZYA/+UBOyi9Y0axEp1L6bR42iC1tCt4kawpm+Lye4aRXdbKo/EBP6wSUiOqQ3+LLLmnaSj
 saAiYQWUZ3at1hOHrZZRIy3vk38rsheWURu6FQWc4VK2odEFtKD2gvw0s93Q6xbMahcWRP3M
 1TWzBSlO4pbV64nXBJVtqDOOs/jQwYjkdotNWb70a2uYasPz+btS0YvW31m94zedXdKZnJN+
 kP314z1q7Gv8CPfgRTqU3JonIkw69ylk2AGzeRv5oFVhRKgFlv0AEQEAAcLAXwQYAQIACQUC
 Uf4lDAIbDAAKCRCFovawdhq1u8wYB/9kZRnMX5gm0Yq7zdqu/K6o3EfAfYI/ZBOXSgYcb58s
 L1Jy+b3inq5PZQrLn7D9V5DFBBjKwthhKVK/eKCALqYuVvaiBmhHjE02xZoi1g7pvV2kj0z6
 OFtF2IXO4vwtaHQhhIutVd+jjDmnSl0kYCBurOjVFmD2ZCTDQ5/JqEDU26V5i9Dwp9sImDm7
 r3lBgLOKu5uWKoQRHbdxPN8FzoFfxDH+xZKubqGgvEvCsX1CjFRP7/kcGW5TrAb/rNEOG1Ik
 25Qj7mjWFa6sv2jYvV1aIpCK8sKKTPeS0mRfhnqQDYqluBsOQIrSHSHbjCTkuIdSutzJyxpG
 xLY7n9TPT2ug
In-Reply-To: <CAC_UY88wFwye8evg1sZq5j=hqTqmM92xLRMznNHHRUKWiwUPdA@mail.gmail.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="------------DLWOzEhxIKmps0F00GfwUaDz"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--------------DLWOzEhxIKmps0F00GfwUaDz
Content-Type: multipart/mixed; boundary="------------HVY8BCA7aia9C5w4FeggFCz8";
 protected-headers="v1"
From: =?UTF-8?B?UmFmYcWCIFdvamR5xYJh?= <omeg@invisiblethingslab.com>
To: Owen Smith <owen.smith@citrix.com>
Cc: win-pv-devel@lists.xenproject.org
Message-ID: <bf6b6df0-aa58-44d0-9c59-9a5b022b5620@invisiblethingslab.com>
Subject: Re: [PATCH] Fix double-free on error condition in
 GnttabPermitForeignAccess()
References: <20240307085031.1121-1-omeg@invisiblethingslab.com>
 <CAC_UY88wFwye8evg1sZq5j=hqTqmM92xLRMznNHHRUKWiwUPdA@mail.gmail.com>
In-Reply-To: <CAC_UY88wFwye8evg1sZq5j=hqTqmM92xLRMznNHHRUKWiwUPdA@mail.gmail.com>

--------------HVY8BCA7aia9C5w4FeggFCz8
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64

T24gNyBtYXIgMjAyNCAxMDozNiwgT3dlbiBTbWl0aCB3cm90ZToNCj4gSXMgdGhlcmUgb3Ro
ZXIgY2xlYW51cCBuZWVkZWQgaW4gb3RoZXIgZmFpbCBjYXNlcz8NCj4gQ29udGV4dC0+R3Jh
bnRzLCBDb250ZXh0LT5LZXJuZWxWYSwgQ29udGV4dC0+TWRsIGFyZSBhbGxvY2F0ZWQgYW5k
IHBhZ2VzIA0KPiBjb3VsZCBuZWVkIHJldm9raW5nIGFjY2VzcyBkdXJpbmcgdGhlIGNsZWFu
dXAgb3BlcmF0aW9uIGluIGEgZmFpbHVyZSBjYXNlLg0KPiANCj4gQ2FsbGluZ8KgR250dGFi
U3RvcFNoYXJpbmcoKSBzaG91bGQgZG8gdGhpcywgYnV0IGl0c8Kgbm90IGNhbGxlZCBpbiB0
aGUgDQo+IGNhc2Ugd2hlcmXCoEdudHRhYlBlcm1pdEZvcmVpZ25BY2Nlc3MgcmV0dXJucyBh
IGZhaWx1cmUgY29kZS4NCj4gDQo+IE93ZW4NCj4gDQpHb29kIGNhdGNoLCB0aGUgR250dGFi
U3RvcFNoYXJpbmcoKSBjYWxsIHNob3VsZCByZW1haW4gdGhlcmUgdG8gY2xlYW4gdXAgDQp0
aGUgY29udGVudHMgb2YgdGhlIGNvbnRleHQuIEknbGwgc2VuZCB0aGUgdXBkYXRlZCB2ZXJz
aW9uLg0KDQo+IE9uIFRodSwgTWFyIDcsIDIwMjQgYXQgODo1MeKAr0FNIFJhZmHFgiBXb2pk
ecWCYSANCj4gPG9tZWdAaW52aXNpYmxldGhpbmdzbGFiLmNvbSA8bWFpbHRvOm9tZWdAaW52
aXNpYmxldGhpbmdzbGFiLmNvbT4+IHdyb3RlOg0KPiANCj4gICAgIFhFTklGQUNFX0dOVFRB
Ql9DT05URVhUIGFzc29jaWF0ZWQgd2l0aCB0aGUgcmVxdWVzdCB3YXMgaW5jb3JyZWN0bHkN
Cj4gICAgIGZyZWVkDQo+ICAgICBieSBHbnR0YWJQZXJtaXRGb3JlaWduQWNjZXNzKCkgd2hl
biBhIGZhaWx1cmUgb2NjdXJlZC4gVGhlIGNvbnRleHQNCj4gICAgIGlzIGFsc28NCj4gICAg
IGZyZWVkIGJ5IHRoZSBwYXJlbnQgZnVuY3Rpb24sIElvY3RsR250dGFiUGVybWl0Rm9yZWln
bkFjY2VzcygpLA0KPiAgICAgd2hpY2ggbGVkDQo+ICAgICB0byBhIGRvdWJsZS1mcmVlIGFu
ZCBrZXJuZWwgaGVhcCBjb3JydXB0aW9uLg0KPiANCj4gICAgIFNpZ25lZC1vZmYtYnk6IFJh
ZmHFgiBXb2pkecWCYSA8b21lZ0BpbnZpc2libGV0aGluZ3NsYWIuY29tDQo+ICAgICA8bWFp
bHRvOm9tZWdAaW52aXNpYmxldGhpbmdzbGFiLmNvbT4+DQo+ICAgICAtLS0NCj4gICAgICDC
oHNyYy94ZW5pZmFjZS9pb2N0bF9nbnR0YWIuYyB8IDcgLS0tLS0tLQ0KPiAgICAgIMKgMSBm
aWxlIGNoYW5nZWQsIDcgZGVsZXRpb25zKC0pDQo+IA0KPiAgICAgZGlmZiAtLWdpdCBhL3Ny
Yy94ZW5pZmFjZS9pb2N0bF9nbnR0YWIuYyBiL3NyYy94ZW5pZmFjZS9pb2N0bF9nbnR0YWIu
Yw0KPiAgICAgaW5kZXggOGFiMjA5OS4uMDgzYzI5ZCAxMDA2NDQNCj4gICAgIC0tLSBhL3Ny
Yy94ZW5pZmFjZS9pb2N0bF9nbnR0YWIuYw0KPiAgICAgKysrIGIvc3JjL3hlbmlmYWNlL2lv
Y3RsX2dudHRhYi5jDQo+ICAgICBAQCAtMzAzLDEzICszMDMsNiBAQCBmYWlsMjoNCj4gDQo+
ICAgICAgwqBmYWlsMToNCj4gICAgICDCoCDCoCDCoEVycm9yKCJGYWlsMVxuIik7DQo+ICAg
ICAtwqAgwqAgR250dGFiU3RvcFNoYXJpbmcoRmRvLCBDb250ZXh0LCBQYWdlKTsNCj4gICAg
IC0NCj4gICAgIC3CoCDCoCBpZiAoQ29udGV4dCAhPSBOVUxMKSB7DQo+ICAgICAtwqAgwqAg
wqAgwqAgUnRsWmVyb01lbW9yeShDb250ZXh0LCBzaXplb2YoKkNvbnRleHQpKTsNCj4gICAg
IC3CoCDCoCDCoCDCoCBfX0ZyZWVQb29sV2l0aFRhZyhDb250ZXh0LCBYRU5JRkFDRV9QT09M
X1RBRyk7DQo+ICAgICAtwqAgwqAgfQ0KPiAgICAgLQ0KPiAgICAgIMKgIMKgIMKgcmV0dXJu
IFN0YXR1czsNCj4gICAgICDCoH0NCj4gDQo+ICAgICAtLSANCj4gICAgIDIuNDMuMC53aW5k
b3dzLjENCj4gDQo+IA0KDQotLSANClJhZmHFgiBXb2pkecWCYQ0KSW52aXNpYmxlIFRoaW5n
cyBMYWINCg==

--------------HVY8BCA7aia9C5w4FeggFCz8--

--------------DLWOzEhxIKmps0F00GfwUaDz
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature.asc"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEVwOm02mHaIsSbeWChaL2sHYatbsFAmXpkbwFAwAAAAAACgkQhaL2sHYatbuB
6gf+K6TL8kgKudgDtwlvr1o9boy+PUOL+NtKhUyVo10jJqYGLAuFfyXyyb9ADRo52HYZM6tyeE4v
K2X/LpqrxbLTNV1MJb4Ru0o0cHsTihJV9HAR8wVHjQubSJpiLT6sM5Xhyf0YVsDbTUgWHr/6085i
mVUPImkFjghJj8DyfmVPw5AEVQxo991EHl+CNxFPbX1bY7CKuAztNpx7NkN1ASRM8PMNX+dE8eyk
6BtxKgIhnyWh6Dlc2PE4Cehx69qPYxMt1E0dMF53fqd6AqM5LB3m+7mgl/30jOzdJxdwgsT4qdBz
aUChvmY7eNoA7sDqeZPta5tDIsj0L3HsGwVJ5d8WiQ==
=hKfI
-----END PGP SIGNATURE-----

--------------DLWOzEhxIKmps0F00GfwUaDz--


From win-pv-devel-bounces@lists.xenproject.org Thu Mar 07 10:19:32 2024
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Thu, 07 Mar 2024 10:19:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.689727.1074964 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1riAqY-0002M5-OR; Thu, 07 Mar 2024 10:19:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 689727.1074964; Thu, 07 Mar 2024 10:19:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1riAqY-0002Ly-LV; Thu, 07 Mar 2024 10:19:30 +0000
Received: by outflank-mailman (input) for mailman id 689727;
 Thu, 07 Mar 2024 10:19:28 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kUX/=KN=invisiblethingslab.com=omeg@srs-se1.protection.inumbo.net>)
 id 1riAqW-0002Ls-TJ
 for win-pv-devel@lists.xenproject.org; Thu, 07 Mar 2024 10:19:28 +0000
Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com
 [64.147.123.19]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2c76f654-dc6c-11ee-a1ee-f123f15fe8a2;
 Thu, 07 Mar 2024 11:19:26 +0100 (CET)
Received: from compute2.internal (compute2.nyi.internal [10.202.2.46])
 by mailout.west.internal (Postfix) with ESMTP id 39C7B32002E8;
 Thu,  7 Mar 2024 05:19:24 -0500 (EST)
Received: from mailfrontend2 ([10.202.2.163])
 by compute2.internal (MEProxy); Thu, 07 Mar 2024 05:19:24 -0500
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu,
 7 Mar 2024 05:19:22 -0500 (EST)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2c76f654-dc6c-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:in-reply-to:message-id:mime-version:references:reply-to:subject
	:subject:to:to; s=fm1; t=1709806763; x=1709893163; bh=fub1h/MVX/
	xcYgwCdvkrQgUhQ303xdDPBIooI5SFy7Q=; b=CeMaMh4Yx4yrFU7U5z6wlrBrB9
	3qWoEVAwicN7KIRi3KNKcAMMYG365mOWkE7R4r9oGiBsTLKmY7MT2KJA31ZSTXy/
	ol5Qp2DrC0Bhf4NIjwhR+D76W/OiBSQPYX/YN46r+3ojrnrWbZyirXvZGqYaOd7j
	7DZoevCT9ZadT2w3E+AClobEbzNfHXeuHmAYgfkPl3nHRsropfZSZR0EYG3jq5nU
	TkqviybjDaB1d45HCNMNUpY45CbweGXWAlCyHcQIgMH47zg25HGttRCAgK9VjsWW
	qcgfeU8Gt37ie2v5SQ/g/xOAXII/Incznm8cITt5USwp0yNX7362py0h2ofw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1709806763; x=
	1709893163; bh=fub1h/MVX/xcYgwCdvkrQgUhQ303xdDPBIooI5SFy7Q=; b=a
	kQKj5AcNcc5B/E5AhRT34TEkrXhaHq/P0/WSyz8CTYJWUMTIfreIzMW97v9puTZ/
	t0iZFPmByJsjiaAyeBymgIJQW0HHu9OCFSwPMiA+xGDkvjvmurw2vttNzb+kQkGx
	/RD7YG3FtFJQzcOrWBjONct4702CpLj7I08iCkJy96iuL52aqF+MktZK4QKX+Dfm
	qrUWP2zK6OjdkqkA1UYfABVok7Q05AS5dw59teKn5m48ElydCXZt0ZJDOh9zdT+n
	qHn+XvrfzM8mwZbjfIDiv5ZZnAzicGUH2djo0I1oNRW0lQpE6oNBgBabSQVoUCjr
	4XYgoFoCQvmMGG+TFI8rQ==
X-ME-Sender: <xms:q5TpZS4TS9lXJoK6OA8YY6DmYfG_kXaiteauBOIPyltGWNtRxg0tHg>
    <xme:q5TpZb6akNLGv-u6VYyznQrs6-o1q9eOvlzq5Uw0tAUZZkM48JeJRlC_ZuiIPFlHW
    SHRJs6Gpm9FC-g>
X-ME-Received: <xmr:q5TpZRfvIPX_HMCKaBfuvPFaClJkb4_0lMBVJEFwsw0-BaHQtw0vh9CIvFRM0FU3XTTbqXuUWLlVLTr9skD4B0glG70fNjHfCKf-nJb-tMQ>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrieefgddugecutefuodetggdotefrodftvf
    curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu
    uegrihhlohhuthemuceftddtnecunecujfgurhephffvvefufffkofgjfhggtgfgsehtke
    ertdertdejnecuhfhrohhmpeftrghfrghlucghohhjugihlhgruceoohhmvghgsehinhhv
    ihhsihgslhgvthhhihhnghhslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhephefhve
    dvtedvkeekgeelhffhieejteeuvdekjeejuedtveegjeelvedvgfethedtnecuvehluhhs
    thgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepohhmvghgsehinhhvih
    hsihgslhgvthhhihhnghhslhgrsgdrtghomh
X-ME-Proxy: <xmx:q5TpZfLi65WSc_B8cuG-7to-l0E_6t4yOBIDIbzcekAfIFHvQdagUA>
    <xmx:q5TpZWLRsBLCBebwFHkcDPUNJ_qw8NupzhTUHO7bKuWJD99SBjjLrQ>
    <xmx:q5TpZQzPpVRlFMEj8umm35pb484Btv2TNSpHAP2znPx2NJNLUtIm2g>
    <xmx:q5TpZVkn-sOhF3Y6goYPhxm81YWw1fizRdS2pl6dFi67WTcxmwlLZw>
Feedback-ID: i409c4082:Fastmail
From: =?UTF-8?q?Rafa=C5=82=20Wojdy=C5=82a?= <omeg@invisiblethingslab.com>
To: win-pv-devel@lists.xenproject.org
Cc: =?UTF-8?q?Rafa=C5=82=20Wojdy=C5=82a?= <omeg@invisiblethingslab.com>
Subject: [PATCH] Fix double-free on error condition in GnttabPermitForeignAccess()
Date: Thu,  7 Mar 2024 11:19:01 +0100
Message-ID: <20240307101901.1137-1-omeg@invisiblethingslab.com>
X-Mailer: git-send-email 2.43.0.windows.1
In-Reply-To: <bf6b6df0-aa58-44d0-9c59-9a5b022b5620@invisiblethingslab.com>
References: <bf6b6df0-aa58-44d0-9c59-9a5b022b5620@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

XENIFACE_GNTTAB_CONTEXT associated with the request was incorrectly freed
by GnttabPermitForeignAccess() when a failure occured. The context is also
freed by the parent function, IoctlGnttabPermitForeignAccess(), which led
to a double-free and kernel heap corruption.

GnttabStopSharing() as the final step in the failure path takes care
of cleaning up the contents of the (possibly partially initialized) context.

Signed-off-by: Rafał Wojdyła <omeg@invisiblethingslab.com>
---
 src/xeniface/ioctl_gnttab.c | 6 ------
 1 file changed, 6 deletions(-)

diff --git a/src/xeniface/ioctl_gnttab.c b/src/xeniface/ioctl_gnttab.c
index 8ab2099..026f29b 100644
--- a/src/xeniface/ioctl_gnttab.c
+++ b/src/xeniface/ioctl_gnttab.c
@@ -304,12 +304,6 @@ fail2:
 fail1:
     Error("Fail1\n");
     GnttabStopSharing(Fdo, Context, Page);
-
-    if (Context != NULL) {
-        RtlZeroMemory(Context, sizeof(*Context));
-        __FreePoolWithTag(Context, XENIFACE_POOL_TAG);
-    }
-
     return Status;
 }
 
-- 
2.43.0.windows.1



From win-pv-devel-bounces@lists.xenproject.org Tue Mar 12 07:23:58 2024
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Tue, 12 Mar 2024 07:23:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.691647.1077864 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rjwUN-0000Vu-Gs; Tue, 12 Mar 2024 07:23:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 691647.1077864; Tue, 12 Mar 2024 07:23:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rjwUN-0000Vn-Dq; Tue, 12 Mar 2024 07:23:55 +0000
Received: by outflank-mailman (input) for mailman id 691647;
 Tue, 12 Mar 2024 07:23:53 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=B9MD=KS=cloud.com=owen.smith@srs-se1.protection.inumbo.net>)
 id 1rjwUL-0000Vh-5K
 for win-pv-devel@lists.xenproject.org; Tue, 12 Mar 2024 07:23:53 +0000
Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com
 [2a00:1450:4864:20::336])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 78563c18-e041-11ee-afdd-a90da7624cb6;
 Tue, 12 Mar 2024 08:23:50 +0100 (CET)
Received: by mail-wm1-x336.google.com with SMTP id
 5b1f17b1804b1-413383969efso1369845e9.1
 for <win-pv-devel@lists.xenproject.org>; Tue, 12 Mar 2024 00:23:49 -0700 (PDT)
Received: from localhost.localdomain
 (host86-169-131-93.range86-169.btcentralplus.com. [86.169.131.93])
 by smtp.gmail.com with ESMTPSA id
 bh27-20020a05600c3d1b00b004132901d73asm6649020wmb.46.2024.03.12.00.23.46
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 12 Mar 2024 00:23:47 -0700 (PDT)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 78563c18-e041-11ee-afdd-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1710228228; x=1710833028; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=Co+MAENA+MPEWQFqFqHErtFzh4pXTPOsLiLVmb1zZwc=;
        b=e3XmHpgk9qXnA+t46KaSxJ/0retUWPcqDbAiQhlQS/8UvkXQyNuqZ5PtK6HnsLdhzG
         uIS0qjhmv5IOW5LdfBeEiMKh3BwLKs7ETdbXn+nAg8wPNqbd71JgbEBv5nyDShz3xekS
         rHCazFlREOnpGH0yITOuKHNV9m3/aS18OTQds=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1710228228; x=1710833028;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=Co+MAENA+MPEWQFqFqHErtFzh4pXTPOsLiLVmb1zZwc=;
        b=e+b27RSBjWRqmGxMWmMbZGSoX3JKj84JmTodqDai3Ny7WgLK4Fqu2utsjZtHZC2AnO
         k7zQe7o4D8AmnWIefsZ6Jyzx0zoE6+TTWevDlBCgi60D5stwNettU0l52MnAf4rrP74W
         Xd1CW1UZkv1QDGgUZ3U39dZ9Xk1oQ3HkTRXTFcmTszmHJ/jgA/CqfpXtIdRXiqG0XcQD
         Kca7Z+GmgCMRCJUBsMI0/Q9o0rNe9kJvW001GELH03ZISIKZAQejQnckL8tqi1leaP7C
         rJCGFDzsVcQV8IhqqLQ4OEgCUV9k2UibzAWwnweI+2PO9eRg/w0TpzK3oXLVZMT1ZTRb
         is1Q==
X-Gm-Message-State: AOJu0YwFplv7lTZbArKWDsZoZ8EDrjCFPryAof/QpN320cuz6tgxcCaf
	z/WLetMhLAYNfnRg88NX4oe4ONG1Z1B9pXI323NpCAw7sBcQGQEDXajUZKF32UKFXq5dbR0GeG8
	WFQ==
X-Google-Smtp-Source: AGHT+IE51v1QMZj57I9XzZGjuff6F1pvF6ljVpxV/EJbdWVIqJJbUCJWTUJVH1ycge9YYiL6MrkYXQ==
X-Received: by 2002:a05:600c:3514:b0:413:28ae:ebcb with SMTP id h20-20020a05600c351400b0041328aeebcbmr5477010wmq.3.1710228227574;
        Tue, 12 Mar 2024 00:23:47 -0700 (PDT)
From: Owen Smith <owen.smith@cloud.com>
To: win-pv-devel@lists.xenproject.org
Cc: Martin Harvey <Martin.Harvey@citrix.com>,
	Martin Harvey <martin.harvey@citrix.com>,
	Owen Smith <owen.smith@cloud.com>
Subject: [PATCH] Asynchronous power handling.
Date: Tue, 12 Mar 2024 07:23:33 +0000
Message-ID: <20240312072333.787-1-owen.smith@cloud.com>
X-Mailer: git-send-email 2.41.0.windows.3
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Martin Harvey <Martin.Harvey@citrix.com>

Use completion routines and IO_WORKITEMs to handle power state transitions,
allowing execution to run at PASSIVE_LEVEL without blocking the current thread,
and avoid blocking all power IRPs while another is executing (power IRPs should
be synchronized by the power manager, but this doesnt appear to be the case in
all situations).

Signed-off-by: Martin Harvey <martin.harvey@citrix.com>

Refactored
Signed-off-by: Owen Smith <owen.smith@cloud.com>

---
 src/xenvif/fdo.c | 803 +++++++++++++++++++++++++++++------------------
 src/xenvif/pdo.c | 263 +++++++---------
 2 files changed, 608 insertions(+), 458 deletions(-)

diff --git a/src/xenvif/fdo.c b/src/xenvif/fdo.c
index 451d582..7fa2d04 100644
--- a/src/xenvif/fdo.c
+++ b/src/xenvif/fdo.c
@@ -81,9 +81,9 @@ struct _XENVIF_FDO {
     ULONG                       Usage[DeviceUsageTypeDumpFile + 1];
     BOOLEAN                     NotDisableable;
 
-    PXENVIF_THREAD              SystemPowerThread;
+    PIO_WORKITEM                SystemPowerWorkItem;
     PIRP                        SystemPowerIrp;
-    PXENVIF_THREAD              DevicePowerThread;
+    PIO_WORKITEM                DevicePowerWorkItem;
     PIRP                        DevicePowerIrp;
 
     CHAR                        VendorName[MAXNAMELEN];
@@ -2168,6 +2168,68 @@ FdoDispatchPnp(
     return status;
 }
 
+__drv_functionClass(IO_WORKITEM_ROUTINE)
+__drv_sameIRQL
+static VOID
+FdoDevicePowerUpWorker(
+    IN  PDEVICE_OBJECT  DeviceObject,
+    IN  PVOID           Context
+    )
+{
+    PXENVIF_FDO         Fdo = (PXENVIF_FDO) Context;
+    PIRP                Irp;
+
+    UNREFERENCED_PARAMETER(DeviceObject);
+
+    Irp = InterlockedExchangePointer(&Fdo->DevicePowerIrp, NULL);
+    ASSERT(Irp != NULL);
+
+    (VOID) FdoD3ToD0(Fdo);
+
+    /* Cannot change Irp->IoStatus */
+    /* Continue completion chain */
+    IoCompleteRequest(Irp, IO_NO_INCREMENT);
+}
+
+__drv_functionClass(IO_COMPLETION_ROUTINE)
+__drv_sameIRQL
+static NTSTATUS
+FdoSetDevicePowerUpComplete(
+    IN  PDEVICE_OBJECT  DeviceObject,
+    IN  PIRP            Irp,
+    IN  PVOID           Context
+    )
+{
+    PXENVIF_FDO         Fdo = (PXENVIF_FDO) Context;
+    PIO_STACK_LOCATION  StackLocation;
+    DEVICE_POWER_STATE  DeviceState;
+    PVOID               Exchange;
+
+    UNREFERENCED_PARAMETER(DeviceObject);
+
+    StackLocation = IoGetCurrentIrpStackLocation(Irp);
+    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
+
+    /* Already marked pending by us */
+    Info("%s -> %s\n",
+         PowerDeviceStateName(__FdoGetDevicePowerState(Fdo)),
+         PowerDeviceStateName(DeviceState));
+
+    /* Don't worry about IRP IoStatus */
+    ASSERT3U(DeviceState, ==, PowerDeviceD0);
+
+    Exchange = InterlockedExchangePointer(&Fdo->DevicePowerIrp, Irp);
+    ASSERT(Exchange == NULL);
+
+    IoQueueWorkItem(Fdo->DevicePowerWorkItem,
+                    FdoDevicePowerUpWorker,
+                    DelayedWorkQueue,
+                    Fdo);
+
+    /* Stop completion chain in all circumstances. */
+    return STATUS_MORE_PROCESSING_REQUIRED;
+}
+
 static FORCEINLINE NTSTATUS
 __FdoSetDevicePowerUp(
     IN  PXENVIF_FDO     Fdo,
@@ -2176,7 +2238,6 @@ __FdoSetDevicePowerUp(
 {
     PIO_STACK_LOCATION  StackLocation;
     DEVICE_POWER_STATE  DeviceState;
-    NTSTATUS            status;
 
     Trace("====>\n");
 
@@ -2185,25 +2246,41 @@ __FdoSetDevicePowerUp(
 
     ASSERT3U(DeviceState, <,  __FdoGetDevicePowerState(Fdo));
 
-    status = FdoForwardIrpSynchronously(Fdo, Irp);
-    if (!NT_SUCCESS(status))
-        goto done;
+    IoMarkIrpPending(Irp);
+    IoCopyCurrentIrpStackLocationToNext(Irp);
+    IoSetCompletionRoutine(Irp,
+                           FdoSetDevicePowerUpComplete,
+                           Fdo,
+                           TRUE,
+                           TRUE,
+                           TRUE);
 
-    Info("%s: %s -> %s\n",
-         __FdoGetName(Fdo),
-         PowerDeviceStateName(__FdoGetDevicePowerState(Fdo)),
-         PowerDeviceStateName(DeviceState));
+    (VOID) IoCallDriver(Fdo->LowerDeviceObject, Irp);
 
-    ASSERT3U(DeviceState, ==, PowerDeviceD0);
-    status = FdoD3ToD0(Fdo);
-    ASSERT(NT_SUCCESS(status));
+    return STATUS_PENDING;
+}
 
-done:
-    Irp->IoStatus.Status = status;
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
+__drv_functionClass(IO_WORKITEM_ROUTINE)
+__drv_sameIRQL
+static VOID
+FdoDevicePowerDownWorker(
+    IN  PDEVICE_OBJECT  DeviceObject,
+    IN  PVOID           Context
+    )
+{
+    PXENVIF_FDO         Fdo = (PXENVIF_FDO) Context;
+    PIRP                Irp;
 
-    Trace("<==== (%08x)\n", status);
-    return status;
+    UNREFERENCED_PARAMETER(DeviceObject);
+
+    Irp = InterlockedExchangePointer(&Fdo->DevicePowerIrp, NULL);
+    ASSERT(Irp != NULL);
+
+    FdoD0ToD3(Fdo);
+
+    /* We are on dispatch path here. */
+    IoCopyCurrentIrpStackLocationToNext(Irp);
+    IoCallDriver(Fdo->LowerDeviceObject, Irp);
 }
 
 static FORCEINLINE NTSTATUS
@@ -2221,18 +2298,32 @@ __FdoSetDevicePowerDown(
 
     ASSERT3U(DeviceState, >,  __FdoGetDevicePowerState(Fdo));
 
-    Info("%s: %s -> %s\n",
-         __FdoGetName(Fdo),
+    Info("%s -> %s\n",
          PowerDeviceStateName(__FdoGetDevicePowerState(Fdo)),
          PowerDeviceStateName(DeviceState));
 
     ASSERT3U(DeviceState, ==, PowerDeviceD3);
 
-    if (__FdoGetDevicePowerState(Fdo) == PowerDeviceD0)
-        FdoD0ToD3(Fdo);
+    /* All handled in dispatch, no extra error handling */
+    if (__FdoGetDevicePowerState(Fdo) != PowerDeviceD0) {
+        /* Not marked pending, can skip */
+        IoSkipCurrentIrpStackLocation(Irp);
+        status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
+    } else {
+        PVOID           Exchange;
 
-    IoSkipCurrentIrpStackLocation(Irp);
-    status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
+        IoMarkIrpPending(Irp);
+
+        Exchange = InterlockedExchangePointer(&Fdo->DevicePowerIrp, Irp);
+        ASSERT(Exchange == NULL);
+
+        IoQueueWorkItem(Fdo->DevicePowerWorkItem,
+                        FdoDevicePowerDownWorker,
+                        DelayedWorkQueue,
+                        Fdo);
+
+        status = STATUS_PENDING;
+    }
 
     return status;
 }
@@ -2253,11 +2344,9 @@ __FdoSetDevicePower(
     PowerAction = StackLocation->Parameters.Power.ShutdownType;
 
     Trace("====> (%s:%s)\n",
-          PowerDeviceStateName(DeviceState), 
+          PowerDeviceStateName(DeviceState),
           PowerActionName(PowerAction));
 
-    ASSERT3U(PowerAction, <, PowerActionShutdown);
-
     if (DeviceState == __FdoGetDevicePowerState(Fdo)) {
         IoSkipCurrentIrpStackLocation(Irp);
         status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
@@ -2271,7 +2360,7 @@ __FdoSetDevicePower(
 
 done:
     Trace("<==== (%s:%s)(%08x)\n",
-          PowerDeviceStateName(DeviceState), 
+          PowerDeviceStateName(DeviceState),
           PowerActionName(PowerAction),
           status);
     return status;
@@ -2279,8 +2368,8 @@ done:
 
 __drv_functionClass(REQUEST_POWER_COMPLETE)
 __drv_sameIRQL
-VOID
-__FdoRequestSetDevicePower(
+static VOID
+FdoRequestSetDevicePowerUpComplete(
     IN  PDEVICE_OBJECT      DeviceObject,
     IN  UCHAR               MinorFunction,
     IN  POWER_STATE         PowerState,
@@ -2288,47 +2377,130 @@ __FdoRequestSetDevicePower(
     IN  PIO_STATUS_BLOCK    IoStatus
     )
 {
-    PKEVENT                 Event = Context;
+    PIRP                    Irp = (PIRP) Context;
 
     UNREFERENCED_PARAMETER(DeviceObject);
     UNREFERENCED_PARAMETER(MinorFunction);
     UNREFERENCED_PARAMETER(PowerState);
+    UNREFERENCED_PARAMETER(IoStatus);
 
-    ASSERT(NT_SUCCESS(IoStatus->Status));
-
-    KeSetEvent(Event, IO_NO_INCREMENT, FALSE);
+    /* Although can change Irp->IoStatus for pended IRP, drivers should not fail this. */
+    IoCompleteRequest(Irp, IO_NO_INCREMENT);
 }
 
+__drv_functionClass(IO_WORKITEM_ROUTINE)
+__drv_sameIRQL
 static VOID
-FdoRequestSetDevicePower(
-    IN  PXENVIF_FDO         Fdo,
-    IN  DEVICE_POWER_STATE  DeviceState
+FdoSystemPowerUpWorker(
+    IN  PDEVICE_OBJECT  DeviceObject,
+    IN  PVOID           Context
     )
 {
-    POWER_STATE             PowerState;
-    KEVENT                  Event;
-    NTSTATUS                status;
+    PXENVIF_FDO         Fdo = (PXENVIF_FDO) Context;
+    PIRP                Irp;
+    PIO_STACK_LOCATION  StackLocation;
+    SYSTEM_POWER_STATE  SystemState;
+    POWER_STATE         PowerState;
+    NTSTATUS            status;
 
-    Trace("%s\n", PowerDeviceStateName(DeviceState));
+    UNREFERENCED_PARAMETER(DeviceObject);
 
-    ASSERT3U(KeGetCurrentIrql(), ==, PASSIVE_LEVEL);
+    Irp = InterlockedExchangePointer(&Fdo->SystemPowerIrp, NULL);
+    ASSERT(Irp != NULL);
 
-    PowerState.DeviceState = DeviceState;
-    KeInitializeEvent(&Event, NotificationEvent, FALSE);
+    StackLocation = IoGetCurrentIrpStackLocation(Irp);;
+    SystemState = StackLocation->Parameters.Power.State.SystemState;
+
+    Info("%s -> %s\n",
+         PowerSystemStateName(__FdoGetSystemPowerState(Fdo)),
+         PowerSystemStateName(SystemState));
+
+    __FdoSetSystemPowerState(Fdo, PowerSystemHibernate);
+    FdoS4ToS3(Fdo);
+    __FdoSetSystemPowerState(Fdo, SystemState);
+
+    PowerState.DeviceState = Fdo->LowerDeviceCapabilities.DeviceState[SystemState];
 
     status = PoRequestPowerIrp(Fdo->LowerDeviceObject,
                                IRP_MN_SET_POWER,
                                PowerState,
-                               __FdoRequestSetDevicePower,
-                               &Event,
+                               FdoRequestSetDevicePowerUpComplete,
+                               Irp,
                                NULL);
-    ASSERT(NT_SUCCESS(status));
 
-    (VOID) KeWaitForSingleObject(&Event,
-                                 Executive,
-                                 KernelMode,
-                                 FALSE,
-                                 NULL);
+    /* Whatever happens, we have stopped completion processing for SIrp,
+      to restart SIrp processing, call IoCompleteRequest. */
+    if (!NT_SUCCESS(status))
+        goto fail1;
+
+    return; /* FdoRequestSetDevicePowerUpComplete will complete the request.*/
+
+fail1:
+    Error("fail1 - but continue IRP processing. (%08x)\n", status);
+    /* Although can change Irp->IoStatus for pended IRP, drivers should not fail this. */
+    IoCompleteRequest(Irp, IO_NO_INCREMENT);
+}
+
+__drv_functionClass(IO_COMPLETION_ROUTINE)
+__drv_sameIRQL
+static NTSTATUS
+FdoSetSystemPowerUpComplete(
+    IN  PDEVICE_OBJECT  DeviceObject,
+    IN  PIRP            Irp,
+    IN  PVOID           Context
+    )
+{
+    PXENVIF_FDO         Fdo = (PXENVIF_FDO) Context;
+    PIO_STACK_LOCATION  StackLocation;
+    SYSTEM_POWER_STATE  SystemState;
+    POWER_STATE         PowerState;
+    NTSTATUS            status;
+
+    UNREFERENCED_PARAMETER(DeviceObject);
+
+    /* IRP marked as pending on dispatch path, no need to check pending returned */
+
+    /* Don't worry about IRP IoStatus */
+    StackLocation = IoGetCurrentIrpStackLocation(Irp);
+    SystemState = StackLocation->Parameters.Power.State.SystemState;
+
+    if (SystemState < PowerSystemHibernate &&
+        __FdoGetSystemPowerState(Fdo) >= PowerSystemHibernate) {
+        PVOID           Exchange;
+
+        Exchange = InterlockedExchangePointer(&Fdo->SystemPowerIrp, Irp);
+        ASSERT(Exchange == NULL);
+
+        IoQueueWorkItem(Fdo->SystemPowerWorkItem,
+                        FdoSystemPowerUpWorker,
+                        DelayedWorkQueue,
+                        Fdo);
+    } else {
+        Info("%s -> %s\n",
+             PowerSystemStateName(__FdoGetSystemPowerState(Fdo)),
+             PowerSystemStateName(SystemState));
+
+        __FdoSetSystemPowerState(Fdo, SystemState);
+
+        PowerState.DeviceState = Fdo->LowerDeviceCapabilities.DeviceState[SystemState];
+
+        status = PoRequestPowerIrp(Fdo->LowerDeviceObject,
+                                   IRP_MN_SET_POWER,
+                                   PowerState,
+                                   FdoRequestSetDevicePowerUpComplete,
+                                   Irp,
+                                   NULL);
+        if (!NT_SUCCESS(status))
+            goto fail1;
+    }
+
+    /* Will later complete the IRP (which is marked pending, in worker or 2nd level comp routine.)*/
+    return STATUS_MORE_PROCESSING_REQUIRED;
+
+fail1:
+    Error("fail1 - but continue IRP processing. (%08x)\n", status);
+    /* marked pending on dispatch path, don't change Irp->IoStatus, keep going.*/
+    return STATUS_CONTINUE_COMPLETION;
 }
 
 static FORCEINLINE NTSTATUS
@@ -2337,42 +2509,103 @@ __FdoSetSystemPowerUp(
     IN  PIRP            Irp
     )
 {
-
     PIO_STACK_LOCATION  StackLocation;
     SYSTEM_POWER_STATE  SystemState;
-    DEVICE_POWER_STATE  DeviceState;
-    NTSTATUS            status;
 
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
     SystemState = StackLocation->Parameters.Power.State.SystemState;
 
     ASSERT3U(SystemState, <,  __FdoGetSystemPowerState(Fdo));
 
-    status = FdoForwardIrpSynchronously(Fdo, Irp);
-    if (!NT_SUCCESS(status))
-        goto done;
+    /* IRP already pended, and we want to complete it *after* completion routine. */
+    IoCopyCurrentIrpStackLocationToNext(Irp);
+    IoSetCompletionRoutine(Irp,
+                           FdoSetSystemPowerUpComplete,
+                           Fdo,
+                           TRUE,
+                           TRUE,
+                           TRUE);
 
-    Info("%s: %s -> %s\n",
-         __FdoGetName(Fdo),
-         PowerSystemStateName(__FdoGetSystemPowerState(Fdo)),
-         PowerSystemStateName(SystemState));
+    IoCallDriver(Fdo->LowerDeviceObject, Irp);
 
-    if (SystemState < PowerSystemHibernate &&
-        __FdoGetSystemPowerState(Fdo) >= PowerSystemHibernate) {
-        __FdoSetSystemPowerState(Fdo, PowerSystemHibernate);
-        FdoS4ToS3(Fdo);
-    }
+    return STATUS_PENDING;
+}
+
+__drv_functionClass(IO_WORKITEM_ROUTINE)
+__drv_sameIRQL
+static VOID
+FdoSystemPowerDownWorker(
+    IN  PDEVICE_OBJECT  DeviceObject,
+    IN  PVOID           Context
+    )
+{
+    PXENVIF_FDO         Fdo = (PXENVIF_FDO) Context;
+    PIRP                Irp;
+    PIO_STACK_LOCATION  StackLocation;
+    SYSTEM_POWER_STATE  SystemState;
+
+    UNREFERENCED_PARAMETER(DeviceObject);
+
+    Irp = InterlockedExchangePointer(&Fdo->SystemPowerIrp, NULL);
+    ASSERT(Irp != NULL);
 
+    StackLocation = IoGetCurrentIrpStackLocation(Irp);
+    SystemState = StackLocation->Parameters.Power.State.SystemState;
+
+    __FdoSetSystemPowerState(Fdo, PowerSystemSleeping3);
+    FdoS3ToS4(Fdo);
     __FdoSetSystemPowerState(Fdo, SystemState);
 
-    DeviceState = Fdo->LowerDeviceCapabilities.DeviceState[SystemState];
-    FdoRequestSetDevicePower(Fdo, DeviceState);
+    IoCopyCurrentIrpStackLocationToNext(Irp); /* Irp has been pended */
+    IoCallDriver(Fdo->LowerDeviceObject, Irp);
+}
 
-done:
-    Irp->IoStatus.Status = status;
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
+__drv_functionClass(REQUEST_POWER_COMPLETE)
+__drv_sameIRQL
+static VOID
+FdoRequestSetDevicePowerDownComplete(
+    IN  PDEVICE_OBJECT      DeviceObject,
+    IN  UCHAR               MinorFunction,
+    IN  POWER_STATE         PowerState,
+    IN  PVOID               Context,
+    IN  PIO_STATUS_BLOCK    IoStatus
+    )
+{
+    PIRP                    Irp = (PIRP) Context;
+    PIO_STACK_LOCATION      StackLocation = IoGetCurrentIrpStackLocation(Irp);
+    PDEVICE_OBJECT          UpperDeviceObject = StackLocation->DeviceObject;
+    PXENVIF_DX              Dx = (PXENVIF_DX)UpperDeviceObject->DeviceExtension;
+    PXENVIF_FDO             Fdo = Dx->Fdo;
+    SYSTEM_POWER_STATE      SystemState = StackLocation->Parameters.Power.State.SystemState;
 
-    return status;
+    UNREFERENCED_PARAMETER(DeviceObject);
+    UNREFERENCED_PARAMETER(MinorFunction);
+    UNREFERENCED_PARAMETER(PowerState);
+
+    if (!NT_SUCCESS(IoStatus->Status))
+        Error("fail1 - but continue IRP processing. (%08x)\n", IoStatus->Status);
+
+    Info("%s -> %s\n",
+        PowerSystemStateName(__FdoGetSystemPowerState(Fdo)),
+        PowerSystemStateName(SystemState));
+
+    if (SystemState >= PowerSystemHibernate &&
+        __FdoGetSystemPowerState(Fdo) < PowerSystemHibernate) {
+        PVOID Exchange;
+
+        Exchange = InterlockedExchangePointer(&Fdo->SystemPowerIrp, Irp);
+        ASSERT(Exchange == NULL);
+
+        IoQueueWorkItem(Fdo->DevicePowerWorkItem,
+                        FdoSystemPowerDownWorker,
+                        DelayedWorkQueue,
+                        Fdo);
+    } else {
+        __FdoSetSystemPowerState(Fdo, SystemState);
+
+        IoCopyCurrentIrpStackLocationToNext(Irp); /* Irp has been pended */
+        IoCallDriver(Fdo->LowerDeviceObject, Irp);
+    }
 }
 
 static FORCEINLINE NTSTATUS
@@ -2383,33 +2616,44 @@ __FdoSetSystemPowerDown(
 {
     PIO_STACK_LOCATION  StackLocation;
     SYSTEM_POWER_STATE  SystemState;
-    DEVICE_POWER_STATE  DeviceState;
+    POWER_STATE         PowerState;
     NTSTATUS            status;
 
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
+    BUG_ON(StackLocation->DeviceObject != Fdo->Dx->DeviceObject);
     SystemState = StackLocation->Parameters.Power.State.SystemState;
 
     ASSERT3U(SystemState, >,  __FdoGetSystemPowerState(Fdo));
 
-    DeviceState = Fdo->LowerDeviceCapabilities.DeviceState[SystemState];
-
-    FdoRequestSetDevicePower(Fdo, DeviceState);
+    PowerState.DeviceState = Fdo->LowerDeviceCapabilities.DeviceState[SystemState];
 
-    Info("%s: %s -> %s\n",
-         __FdoGetName(Fdo),
-         PowerSystemStateName(__FdoGetSystemPowerState(Fdo)),
-         PowerSystemStateName(SystemState));
-
-    if (SystemState >= PowerSystemHibernate &&
-        __FdoGetSystemPowerState(Fdo) < PowerSystemHibernate) {
-        __FdoSetSystemPowerState(Fdo, PowerSystemSleeping3);
-        FdoS3ToS4(Fdo);
+    if (SystemState >= PowerSystemShutdown) {
+        /* No DIrp generation, no FDO Powerdown.
+           Legacy shutdown just yanks system power.
+           Best device state for S5 is advertised as D3,
+           which is not *really* the case, but we can't increase it. */
+        IoCopyCurrentIrpStackLocationToNext(Irp); /* Irp has been pended */
+        status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
+    } else {
+        status = PoRequestPowerIrp(Fdo->LowerDeviceObject,
+                                   IRP_MN_SET_POWER,
+                                   PowerState,
+                                   FdoRequestSetDevicePowerDownComplete,
+                                   Irp,
+                                   NULL);
+        if (!NT_SUCCESS(status))
+            goto fail1;
     }
+    /* IRP already marked pending - nothing more to do if awaiting callback. */
 
-    __FdoSetSystemPowerState(Fdo, SystemState);
+    return status;
 
-    IoSkipCurrentIrpStackLocation(Irp);
-    status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
+fail1:
+    /* In theory could change IRP status, but are not supposed to fail this IRP. */
+    Error("fail1 - but continue IRP processing. (%08x)\n", status);
+
+    IoCopyCurrentIrpStackLocationToNext(Irp); /* Irp has been pended */
+    IoCallDriver(Fdo->LowerDeviceObject, Irp);
 
     return status;
 }
@@ -2429,29 +2673,31 @@ __FdoSetSystemPower(
     SystemState = StackLocation->Parameters.Power.State.SystemState;
     PowerAction = StackLocation->Parameters.Power.ShutdownType;
 
+    /* IRP_MN_SET_POWER Setting system power, IRP's *must* be pended. */
+    IoMarkIrpPending(Irp);
+
     Trace("====> (%s:%s)\n",
-          PowerSystemStateName(SystemState), 
+          PowerSystemStateName(SystemState),
           PowerActionName(PowerAction));
 
-    ASSERT3U(PowerAction, <, PowerActionShutdown);
-
     if (SystemState == __FdoGetSystemPowerState(Fdo)) {
-        IoSkipCurrentIrpStackLocation(Irp);
+        IoCopyCurrentIrpStackLocationToNext(Irp); /* Pended, copy not skip */
         status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
 
         goto done;
     }
 
-    status = (SystemState < __FdoGetSystemPowerState(Fdo)) ?
+    status = SystemState < __FdoGetSystemPowerState(Fdo) ?
              __FdoSetSystemPowerUp(Fdo, Irp) :
              __FdoSetSystemPowerDown(Fdo, Irp);
 
 done:
     Trace("<==== (%s:%s)(%08x)\n",
-          PowerSystemStateName(SystemState), 
+          PowerSystemStateName(SystemState),
           PowerActionName(PowerAction),
           status);
-    return status;
+
+    return STATUS_PENDING;
 }
 
 static FORCEINLINE NTSTATUS
@@ -2469,10 +2715,9 @@ __FdoQueryDevicePowerUp(
 
     ASSERT3U(DeviceState, <,  __FdoGetDevicePowerState(Fdo));
 
-    status = FdoForwardIrpSynchronously(Fdo, Irp);
-
-    Irp->IoStatus.Status = status;
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
+    /* N.B. Shutdown / hibernation failures caused by our device state could be vetoed here. */
+    IoSkipCurrentIrpStackLocation(Irp);
+    status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
 
     return status;
 }
@@ -2492,6 +2737,7 @@ __FdoQueryDevicePowerDown(
 
     ASSERT3U(DeviceState, >,  __FdoGetDevicePowerState(Fdo));
 
+    /* N.B. Shutdown / hibernation failures caused by our device state could be vetoed here. */
     IoSkipCurrentIrpStackLocation(Irp);
     status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
 
@@ -2514,11 +2760,9 @@ __FdoQueryDevicePower(
     PowerAction = StackLocation->Parameters.Power.ShutdownType;
 
     Trace("====> (%s:%s)\n",
-          PowerDeviceStateName(DeviceState), 
+          PowerDeviceStateName(DeviceState),
           PowerActionName(PowerAction));
 
-    ASSERT3U(PowerAction, <, PowerActionShutdown);
-
     if (DeviceState == __FdoGetDevicePowerState(Fdo)) {
         IoSkipCurrentIrpStackLocation(Irp);
         status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
@@ -2532,7 +2776,7 @@ __FdoQueryDevicePower(
 
 done:
     Trace("<==== (%s:%s)(%08x)\n",
-          PowerDeviceStateName(DeviceState), 
+          PowerDeviceStateName(DeviceState),
           PowerActionName(PowerAction),
           status);
     return status;
@@ -2540,8 +2784,8 @@ done:
 
 __drv_functionClass(REQUEST_POWER_COMPLETE)
 __drv_sameIRQL
-VOID
-__FdoRequestQueryDevicePower(
+static VOID
+FdoRequestQueryDevicePowerUpComplete(
     IN  PDEVICE_OBJECT      DeviceObject,
     IN  UCHAR               MinorFunction,
     IN  POWER_STATE         PowerState,
@@ -2549,47 +2793,58 @@ __FdoRequestQueryDevicePower(
     IN  PIO_STATUS_BLOCK    IoStatus
     )
 {
-    PKEVENT                 Event = Context;
+    PIRP                    Irp = (PIRP) Context;
 
     UNREFERENCED_PARAMETER(DeviceObject);
     UNREFERENCED_PARAMETER(MinorFunction);
     UNREFERENCED_PARAMETER(PowerState);
 
-    ASSERT(NT_SUCCESS(IoStatus->Status));
+    if (!NT_SUCCESS(IoStatus->Status))
+        Irp->IoStatus.Status = IoStatus->Status;
 
-    KeSetEvent(Event, IO_NO_INCREMENT, FALSE);
+    IoCompleteRequest(Irp, IO_NO_INCREMENT);
 }
 
-static VOID
-FdoRequestQueryDevicePower(
-    IN  PXENVIF_FDO         Fdo,
-    IN  DEVICE_POWER_STATE  DeviceState
+__drv_functionClass(IO_COMPLETION_ROUTINE)
+__drv_sameIRQL
+static NTSTATUS
+FdoQuerySystemPowerUpComplete(
+    IN  PDEVICE_OBJECT  DeviceObject,
+    IN  PIRP            Irp,
+    IN  PVOID           Context
     )
 {
-    POWER_STATE             PowerState;
-    KEVENT                  Event;
-    NTSTATUS                status;
-
-    Trace("%s\n", PowerDeviceStateName(DeviceState));
+    PXENVIF_FDO         Fdo = (PXENVIF_FDO) Context;
+    PIO_STACK_LOCATION  StackLocation;
+    SYSTEM_POWER_STATE  SystemState;
+    POWER_STATE         PowerState;
+    NTSTATUS            status;
 
-    ASSERT3U(KeGetCurrentIrql(), ==, PASSIVE_LEVEL);
+    UNREFERENCED_PARAMETER(DeviceObject);
 
-    PowerState.DeviceState = DeviceState;
-    KeInitializeEvent(&Event, NotificationEvent, FALSE);
+    /* IRP marked as pending on dispatch path. */
+    StackLocation = IoGetCurrentIrpStackLocation(Irp);
+    SystemState = StackLocation->Parameters.Power.State.SystemState;
+    PowerState.DeviceState = Fdo->LowerDeviceCapabilities.DeviceState[SystemState];
 
     status = PoRequestPowerIrp(Fdo->LowerDeviceObject,
                                IRP_MN_QUERY_POWER,
                                PowerState,
-                               __FdoRequestQueryDevicePower,
-                               &Event,
+                               FdoRequestQueryDevicePowerUpComplete,
+                               Irp,
                                NULL);
-    ASSERT(NT_SUCCESS(status));
+    if (!NT_SUCCESS(status))
+        goto fail1;
 
-    (VOID) KeWaitForSingleObject(&Event,
-                                 Executive,
-                                 KernelMode,
-                                 FALSE,
-                                 NULL);
+    /* Will later complete the IRP (which is marked pending, in 2nd level comp routine.)*/
+    return STATUS_MORE_PROCESSING_REQUIRED;
+
+fail1:
+    /* Irp marked as pending on dispatch path, so can change final IoStatus, pass to completion routine above. */
+    Error("fail1 (%08x)\n", status);
+    Irp->IoStatus.Status = status;
+
+    return STATUS_CONTINUE_COMPLETION;
 }
 
 static FORCEINLINE NTSTATUS
@@ -2598,30 +2853,61 @@ __FdoQuerySystemPowerUp(
     IN  PIRP            Irp
     )
 {
-
     PIO_STACK_LOCATION  StackLocation;
     SYSTEM_POWER_STATE  SystemState;
-    DEVICE_POWER_STATE  DeviceState;
-    NTSTATUS            status;
 
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
     SystemState = StackLocation->Parameters.Power.State.SystemState;
 
     ASSERT3U(SystemState, <,  __FdoGetSystemPowerState(Fdo));
 
-    status = FdoForwardIrpSynchronously(Fdo, Irp);
-    if (!NT_SUCCESS(status))
-        goto done;
+    IoMarkIrpPending(Irp); /* Must mark IRP pending because we want to complete it *after* completion routine. */
+    IoCopyCurrentIrpStackLocationToNext(Irp);
+    IoSetCompletionRoutine(Irp,
+                           FdoQuerySystemPowerUpComplete,
+                           Fdo,
+                           TRUE,
+                           TRUE,
+                           TRUE);
+    (VOID) IoCallDriver(Fdo->LowerDeviceObject, Irp);
 
-    DeviceState = Fdo->LowerDeviceCapabilities.DeviceState[SystemState];
+    return STATUS_PENDING;
+}
 
-    FdoRequestQueryDevicePower(Fdo, DeviceState);
+__drv_functionClass(REQUEST_POWER_COMPLETE)
+__drv_sameIRQL
+static VOID
+FdoRequestQueryDevicePowerDownComplete(
+    IN  PDEVICE_OBJECT      DeviceObject,
+    IN  UCHAR               MinorFunction,
+    IN  POWER_STATE         PowerState,
+    IN  PVOID               Context,
+    IN  PIO_STATUS_BLOCK    IoStatus
+    )
+{
+    PIRP                    Irp = (PIRP) Context;
+    PIO_STACK_LOCATION      StackLocation = IoGetCurrentIrpStackLocation(Irp);
+    PDEVICE_OBJECT          UpperDeviceObject = StackLocation->DeviceObject;
+    PXENVIF_DX              Dx = (PXENVIF_DX)UpperDeviceObject->DeviceExtension;
+    PXENVIF_FDO             Fdo = Dx->Fdo;
 
-done:
-    Irp->IoStatus.Status = status;
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
+    UNREFERENCED_PARAMETER(DeviceObject);
+    UNREFERENCED_PARAMETER(MinorFunction);
+    UNREFERENCED_PARAMETER(PowerState);
 
-    return status;
+    if (!NT_SUCCESS(IoStatus->Status))
+        goto fail1;
+
+    IoCopyCurrentIrpStackLocationToNext(Irp); /* Irp has been pended. */
+    IoCallDriver(Fdo->LowerDeviceObject, Irp);
+
+    return;
+
+fail1:
+    Error("fail1 (%08x)\n", IoStatus->Status);
+
+    Irp->IoStatus.Status = IoStatus->Status;
+    IoCompleteRequest(Irp, IO_NO_INCREMENT);
 }
 
 static FORCEINLINE NTSTATUS
@@ -2632,7 +2918,7 @@ __FdoQuerySystemPowerDown(
 {
     PIO_STACK_LOCATION  StackLocation;
     SYSTEM_POWER_STATE  SystemState;
-    DEVICE_POWER_STATE  DeviceState;
+    POWER_STATE         PowerState;
     NTSTATUS            status;
 
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
@@ -2640,12 +2926,26 @@ __FdoQuerySystemPowerDown(
 
     ASSERT3U(SystemState, >,  __FdoGetSystemPowerState(Fdo));
 
-    DeviceState = Fdo->LowerDeviceCapabilities.DeviceState[SystemState];
+    PowerState.DeviceState = Fdo->LowerDeviceCapabilities.DeviceState[SystemState];
 
-    FdoRequestQueryDevicePower(Fdo, DeviceState);
+    status = PoRequestPowerIrp(Fdo->LowerDeviceObject,
+                               IRP_MN_QUERY_POWER,
+                               PowerState,
+                               FdoRequestQueryDevicePowerDownComplete,
+                               Irp,
+                               NULL);
+    if (!NT_SUCCESS(status))
+        goto fail1;
 
-    IoSkipCurrentIrpStackLocation(Irp);
-    status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
+    IoMarkIrpPending(Irp);
+
+    return STATUS_PENDING;
+
+fail1:
+    Error("fail1 (%08x)\n", status);
+
+    Irp->IoStatus.Status = status;
+    IoCompleteRequest(Irp, IO_NO_INCREMENT);
 
     return status;
 }
@@ -2666,11 +2966,9 @@ __FdoQuerySystemPower(
     PowerAction = StackLocation->Parameters.Power.ShutdownType;
 
     Trace("====> (%s:%s)\n",
-          PowerSystemStateName(SystemState), 
+          PowerSystemStateName(SystemState),
           PowerActionName(PowerAction));
 
-    ASSERT3U(PowerAction, <, PowerActionShutdown);
-
     if (SystemState == __FdoGetSystemPowerState(Fdo)) {
         IoSkipCurrentIrpStackLocation(Irp);
         status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
@@ -2684,183 +2982,91 @@ __FdoQuerySystemPower(
 
 done:
     Trace("<==== (%s:%s)(%08x)\n",
-          PowerSystemStateName(SystemState), 
+          PowerSystemStateName(SystemState),
           PowerActionName(PowerAction),
           status);
 
     return status;
 }
 
-static NTSTATUS
-FdoDevicePower(
-    IN  PXENVIF_THREAD  Self,
-    IN  PVOID           Context
+static FORCEINLINE NTSTATUS
+__FdoDevicePower(
+    IN  PXENVIF_FDO     Fdo,
+    IN  PIRP            Irp
     )
 {
-    PXENVIF_FDO         Fdo = Context;
-    PKEVENT             Event;
-
-    Event = ThreadGetEvent(Self);
-
-    for (;;) {
-        PIRP                Irp;
-        PIO_STACK_LOCATION  StackLocation;
-        UCHAR               MinorFunction;
-
-        if (Fdo->DevicePowerIrp == NULL) {
-            (VOID) KeWaitForSingleObject(Event,
-                                         Executive,
-                                         KernelMode,
-                                         FALSE,
-                                         NULL);
-            KeClearEvent(Event);
-        }
-
-        if (ThreadIsAlerted(Self))
-            break;
-
-        Irp = Fdo->DevicePowerIrp;
-
-        if (Irp == NULL)
-            continue;
-
-        Fdo->DevicePowerIrp = NULL;
-        KeMemoryBarrier();
+    PIO_STACK_LOCATION  StackLocation;
+    NTSTATUS            status;
 
-        StackLocation = IoGetCurrentIrpStackLocation(Irp);
-        MinorFunction = StackLocation->MinorFunction;
+    StackLocation = IoGetCurrentIrpStackLocation(Irp);
 
-        switch (StackLocation->MinorFunction) {
-        case IRP_MN_SET_POWER:
-            (VOID) __FdoSetDevicePower(Fdo, Irp);
-            break;
+    switch (StackLocation->MinorFunction) {
+    case IRP_MN_SET_POWER:
+        status = __FdoSetDevicePower(Fdo, Irp);
+        break;
 
-        case IRP_MN_QUERY_POWER:
-            (VOID) __FdoQueryDevicePower(Fdo, Irp);
-            break;
+    case IRP_MN_QUERY_POWER:
+        status = __FdoQueryDevicePower(Fdo, Irp);
+        break;
 
-        default:
-            ASSERT(FALSE);
-            break;
-        }
+    default:
+        IoSkipCurrentIrpStackLocation(Irp);
+        status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
+        break;
     }
 
-    return STATUS_SUCCESS;
+    return status;
 }
 
-static NTSTATUS
-FdoSystemPower(
-    IN  PXENVIF_THREAD  Self,
-    IN  PVOID           Context
+static FORCEINLINE NTSTATUS
+__FdoSystemPower(
+    IN  PXENVIF_FDO     Fdo,
+    IN  PIRP            Irp
     )
 {
-    PXENVIF_FDO         Fdo = Context;
-    PKEVENT             Event;
-
-    Event = ThreadGetEvent(Self);
-
-    for (;;) {
-        PIRP                Irp;
-        PIO_STACK_LOCATION  StackLocation;
-        UCHAR               MinorFunction;
-
-        if (Fdo->SystemPowerIrp == NULL) {
-            (VOID) KeWaitForSingleObject(Event,
-                                         Executive,
-                                         KernelMode,
-                                         FALSE,
-                                         NULL);
-            KeClearEvent(Event);
-        }
-
-        if (ThreadIsAlerted(Self))
-            break;
-
-        Irp = Fdo->SystemPowerIrp;
-
-        if (Irp == NULL)
-            continue;
-
-        Fdo->SystemPowerIrp = NULL;
-        KeMemoryBarrier();
+    PIO_STACK_LOCATION  StackLocation;
+    NTSTATUS            status;
 
-        StackLocation = IoGetCurrentIrpStackLocation(Irp);
-        MinorFunction = StackLocation->MinorFunction;
+    StackLocation = IoGetCurrentIrpStackLocation(Irp);
 
-        switch (StackLocation->MinorFunction) {
-        case IRP_MN_SET_POWER:
-            (VOID) __FdoSetSystemPower(Fdo, Irp);
-            break;
+    switch (StackLocation->MinorFunction) {
+    case IRP_MN_SET_POWER:
+        status = __FdoSetSystemPower(Fdo, Irp);
+        break;
 
-        case IRP_MN_QUERY_POWER:
-            (VOID) __FdoQuerySystemPower(Fdo, Irp);
-            break;
+    case IRP_MN_QUERY_POWER:
+        status = __FdoQuerySystemPower(Fdo, Irp);
+        break;
 
-        default:
-            ASSERT(FALSE);
-            break;
-        }
+    default:
+        IoSkipCurrentIrpStackLocation(Irp);
+        status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
+        break;
     }
 
-    return STATUS_SUCCESS;
+    return status;
 }
 
 static DECLSPEC_NOINLINE NTSTATUS
 FdoDispatchPower(
-    IN  PXENVIF_FDO     Fdo,
+    IN  PXENVIF_FDO   Fdo,
     IN  PIRP            Irp
     )
 {
     PIO_STACK_LOCATION  StackLocation;
-    UCHAR               MinorFunction;
     POWER_STATE_TYPE    PowerType;
-    POWER_ACTION        PowerAction;
     NTSTATUS            status;
 
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    MinorFunction = StackLocation->MinorFunction;
-
-    if (MinorFunction != IRP_MN_QUERY_POWER &&
-        MinorFunction != IRP_MN_SET_POWER) {
-        IoSkipCurrentIrpStackLocation(Irp);
-        status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
-
-        goto done;
-    }
-
     PowerType = StackLocation->Parameters.Power.Type;
-    PowerAction = StackLocation->Parameters.Power.ShutdownType;
-
-    if (PowerAction >= PowerActionShutdown) {
-        IoSkipCurrentIrpStackLocation(Irp);
-        status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
-
-        goto done;
-    }
 
     switch (PowerType) {
     case DevicePowerState:
-        IoMarkIrpPending(Irp);
-
-        ASSERT3P(Fdo->DevicePowerIrp, ==, NULL);
-        Fdo->DevicePowerIrp = Irp;
-        KeMemoryBarrier();
-
-        ThreadWake(Fdo->DevicePowerThread);
-
-        status = STATUS_PENDING;
+        status = __FdoDevicePower(Fdo, Irp);
         break;
 
     case SystemPowerState:
-        IoMarkIrpPending(Irp);
-
-        ASSERT3P(Fdo->SystemPowerIrp, ==, NULL);
-        Fdo->SystemPowerIrp = Irp;
-        KeMemoryBarrier();
-
-        ThreadWake(Fdo->SystemPowerThread);
-
-        status = STATUS_PENDING;
+        status = __FdoSystemPower(Fdo, Irp);
         break;
 
     default:
@@ -2869,7 +3075,6 @@ FdoDispatchPower(
         break;
     }
 
-done:
     return status;
 }
 
@@ -3063,12 +3268,12 @@ FdoCreate(
     Fdo->LowerDeviceObject = IoAttachDeviceToDeviceStack(FunctionDeviceObject,
                                                          PhysicalDeviceObject);
 
-    status = ThreadCreate(FdoSystemPower, Fdo, &Fdo->SystemPowerThread);
-    if (!NT_SUCCESS(status))
+    Fdo->SystemPowerWorkItem = IoAllocateWorkItem(PhysicalDeviceObject);
+    if (Fdo->SystemPowerWorkItem == NULL)
         goto fail3;
 
-    status = ThreadCreate(FdoDevicePower, Fdo, &Fdo->DevicePowerThread);
-    if (!NT_SUCCESS(status))
+    Fdo->DevicePowerWorkItem = IoAllocateWorkItem(PhysicalDeviceObject);
+    if (Fdo->DevicePowerWorkItem == NULL)
         goto fail4;
 
     status = __FdoAcquireLowerBusInterface(Fdo);
@@ -3226,16 +3431,14 @@ fail6:
 fail5:
     Error("fail5\n");
 
-    ThreadAlert(Fdo->DevicePowerThread);
-    ThreadJoin(Fdo->DevicePowerThread);
-    Fdo->DevicePowerThread = NULL;
-    
+    IoFreeWorkItem(Fdo->DevicePowerWorkItem);
+    Fdo->DevicePowerWorkItem = NULL;
+
 fail4:
     Error("fail4\n");
 
-    ThreadAlert(Fdo->SystemPowerThread);
-    ThreadJoin(Fdo->SystemPowerThread);
-    Fdo->SystemPowerThread = NULL;
+    IoFreeWorkItem(Fdo->SystemPowerWorkItem);
+    Fdo->SystemPowerWorkItem = NULL;
     
 fail3:
     Error("fail3\n");
@@ -3311,13 +3514,11 @@ FdoDestroy(
 
     __FdoReleaseLowerBusInterface(Fdo);
 
-    ThreadAlert(Fdo->DevicePowerThread);
-    ThreadJoin(Fdo->DevicePowerThread);
-    Fdo->DevicePowerThread = NULL;
+    IoFreeWorkItem(Fdo->DevicePowerWorkItem);
+    Fdo->DevicePowerWorkItem = NULL;
 
-    ThreadAlert(Fdo->SystemPowerThread);
-    ThreadJoin(Fdo->SystemPowerThread);
-    Fdo->SystemPowerThread = NULL;
+    IoFreeWorkItem(Fdo->SystemPowerWorkItem);
+    Fdo->SystemPowerWorkItem = NULL;
 
     IoDetachDevice(Fdo->LowerDeviceObject);
 
diff --git a/src/xenvif/pdo.c b/src/xenvif/pdo.c
index 0cef2d5..2979f53 100644
--- a/src/xenvif/pdo.c
+++ b/src/xenvif/pdo.c
@@ -68,9 +68,9 @@
 struct _XENVIF_PDO {
     PXENVIF_DX                  Dx;
 
-    PXENVIF_THREAD              SystemPowerThread;
+    PIO_WORKITEM                SystemPowerWorkItem;
     PIRP                        SystemPowerIrp;
-    PXENVIF_THREAD              DevicePowerThread;
+    PIO_WORKITEM                DevicePowerWorkItem;
     PIRP                        DevicePowerIrp;
 
     PXENVIF_FDO                 Fdo;
@@ -2365,28 +2365,31 @@ PdoDispatchPnp(
     return status;
 }
 
-static FORCEINLINE NTSTATUS
-__PdoSetDevicePower(
-    IN  PXENVIF_PDO     Pdo,
-    IN  PIRP            Irp
+__drv_functionClass(IO_WORKITEM_ROUTINE)
+__drv_sameIRQL
+static VOID
+PdoDevicePowerWorker(
+    IN  PDEVICE_OBJECT  DeviceObject,
+    IN  PVOID           Context
     )
 {
+    PXENVIF_PDO         Pdo = (PXENVIF_PDO) Context;
+    PIRP                Irp;
     PIO_STACK_LOCATION  StackLocation;
     DEVICE_POWER_STATE  DeviceState;
     POWER_ACTION        PowerAction;
     NTSTATUS            status;
 
+    UNREFERENCED_PARAMETER(DeviceObject);
+
+    Irp = InterlockedExchangePointer(&Pdo->DevicePowerIrp, NULL);
+    ASSERT(Irp != NULL);
+
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
     DeviceState = StackLocation->Parameters.Power.State.DeviceState;
     PowerAction = StackLocation->Parameters.Power.ShutdownType;
 
-    Trace("====> (%s) (%s:%s)\n",
-          __PdoGetName(Pdo),
-          PowerDeviceStateName(DeviceState), 
-          PowerActionName(PowerAction));
-
-    ASSERT3U(PowerAction, <, PowerActionShutdown);
-
+    status = STATUS_SUCCESS;
     if (__PdoGetDevicePowerState(Pdo) > DeviceState) {
         Trace("%s: POWERING UP: %s -> %s\n",
               __PdoGetName(Pdo),
@@ -2395,7 +2398,6 @@ __PdoSetDevicePower(
 
         ASSERT3U(DeviceState, ==, PowerDeviceD0);
         status = PdoD3ToD0(Pdo);
-        ASSERT(NT_SUCCESS(status));
     } else if (__PdoGetDevicePowerState(Pdo) < DeviceState) {
         Trace("%s: POWERING DOWN: %s -> %s\n",
               __PdoGetName(Pdo),
@@ -2406,77 +2408,71 @@ __PdoSetDevicePower(
         PdoD0ToD3(Pdo);
     }
 
+    /* Cannot fail the IRP at this point, keep going. */
     Irp->IoStatus.Status = STATUS_SUCCESS;
     IoCompleteRequest(Irp, IO_NO_INCREMENT);
 
-    Trace("<==== (%s:%s)\n",
+    Trace("<==== (%s:%s)(%08x)\n",
           PowerDeviceStateName(DeviceState), 
-          PowerActionName(PowerAction));
-
-    return STATUS_SUCCESS;
+          PowerActionName(PowerAction),
+          status);
 }
 
-static NTSTATUS
-PdoDevicePower(
-    IN  PXENVIF_THREAD  Self,
-    IN  PVOID           Context
+static FORCEINLINE NTSTATUS
+__PdoSetDevicePower(
+    IN  PXENVIF_PDO     Pdo,
+    IN  PIRP            Irp
     )
 {
-    PXENVIF_PDO         Pdo = Context;
-    PKEVENT             Event;
-
-    Event = ThreadGetEvent(Self);
-
-    for (;;) {
-        PIRP    Irp;
-
-        if (Pdo->DevicePowerIrp == NULL) {
-            (VOID) KeWaitForSingleObject(Event,
-                                         Executive,
-                                         KernelMode,
-                                         FALSE,
-                                         NULL);
-            KeClearEvent(Event);
-        }
+    PIO_STACK_LOCATION  StackLocation;
+    DEVICE_POWER_STATE  DeviceState;
+    POWER_ACTION        PowerAction;
+    PVOID               Exchange;
 
-        if (ThreadIsAlerted(Self))
-            break;
+    StackLocation = IoGetCurrentIrpStackLocation(Irp);
+    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
+    PowerAction = StackLocation->Parameters.Power.ShutdownType;
 
-        Irp = Pdo->DevicePowerIrp;
+    Trace("====> (%s:%s)\n",
+          PowerDeviceStateName(DeviceState),
+          PowerActionName(PowerAction));
 
-        if (Irp == NULL)
-            continue;
+    IoMarkIrpPending(Irp);
 
-        Pdo->DevicePowerIrp = NULL;
-        KeMemoryBarrier();
+    Exchange = InterlockedExchangePointer(&Pdo->DevicePowerIrp, Irp);
+    ASSERT(Exchange == NULL);
 
-        (VOID) __PdoSetDevicePower(Pdo, Irp);
-    }
+    IoQueueWorkItem(Pdo->DevicePowerWorkItem,
+                    PdoDevicePowerWorker,
+                    DelayedWorkQueue,
+                    Pdo);
 
-    return STATUS_SUCCESS;
+    return STATUS_PENDING;
 }
 
-static FORCEINLINE NTSTATUS
-__PdoSetSystemPower(
-    IN  PXENVIF_PDO         Pdo,
-    IN  PIRP                Irp
+__drv_functionClass(IO_WORKITEM_ROUTINE)
+__drv_sameIRQL
+static VOID
+PdoSystemPowerWorker(
+    IN  PDEVICE_OBJECT  DeviceObject,
+    IN  PVOID           Context
     )
 {
-    PIO_STACK_LOCATION      StackLocation;
-    SYSTEM_POWER_STATE      SystemState;
-    POWER_ACTION            PowerAction;
+    PXENVIF_PDO         Pdo = (PXENVIF_PDO) Context;
+    PIRP                Irp;
+    PIO_STACK_LOCATION  StackLocation;
+    SYSTEM_POWER_STATE  SystemState;
+    POWER_ACTION        PowerAction;
+
+    UNREFERENCED_PARAMETER(DeviceObject);
+
+    Irp = InterlockedExchangePointer(&Pdo->SystemPowerIrp, NULL);
+    ASSERT(Irp != NULL);
 
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
     SystemState = StackLocation->Parameters.Power.State.SystemState;
     PowerAction = StackLocation->Parameters.Power.ShutdownType;
 
-    Trace("====> (%s) (%s:%s)\n",
-          __PdoGetName(Pdo),
-          PowerSystemStateName(SystemState), 
-          PowerActionName(PowerAction));
-
-    ASSERT3U(PowerAction, <, PowerActionShutdown);
-
     if (__PdoGetSystemPowerState(Pdo) > SystemState) {
         if (SystemState < PowerSystemHibernate &&
             __PdoGetSystemPowerState(Pdo) >= PowerSystemHibernate) {
@@ -2488,6 +2484,7 @@ __PdoSetSystemPower(
               __PdoGetName(Pdo),
               PowerSystemStateName(__PdoGetSystemPowerState(Pdo)),
               PowerSystemStateName(SystemState));
+
     } else if (__PdoGetSystemPowerState(Pdo) < SystemState) {
         Trace("%s: POWERING DOWN: %s -> %s\n",
               __PdoGetName(Pdo),
@@ -2509,97 +2506,60 @@ __PdoSetSystemPower(
     Trace("<==== (%s:%s)\n",
           PowerSystemStateName(SystemState), 
           PowerActionName(PowerAction));
-
-    return STATUS_SUCCESS;
 }
 
-static NTSTATUS
-PdoSystemPower(
-    IN  PXENVIF_THREAD  Self,
-    IN  PVOID           Context
+static FORCEINLINE NTSTATUS
+__PdoSetSystemPower(
+    IN  PXENVIF_PDO     Pdo,
+    IN  PIRP            Irp
     )
 {
-    PXENVIF_PDO         Pdo = Context;
-    PKEVENT             Event;
-
-    Event = ThreadGetEvent(Self);
-
-    for (;;) {
-        PIRP    Irp;
-
-        if (Pdo->SystemPowerIrp == NULL) {
-            (VOID) KeWaitForSingleObject(Event,
-                                         Executive,
-                                         KernelMode,
-                                         FALSE,
-                                         NULL);
-            KeClearEvent(Event);
-        }
+    PIO_STACK_LOCATION  StackLocation;
+    SYSTEM_POWER_STATE  SystemState;
+    POWER_ACTION        PowerAction;
+    PVOID               Exchange;
 
-        if (ThreadIsAlerted(Self))
-            break;
+    StackLocation = IoGetCurrentIrpStackLocation(Irp);
+    SystemState = StackLocation->Parameters.Power.State.SystemState;
+    PowerAction = StackLocation->Parameters.Power.ShutdownType;
 
-        Irp = Pdo->SystemPowerIrp;
+    Trace("====> (%s:%s)\n",
+          PowerSystemStateName(SystemState),
+          PowerActionName(PowerAction));
 
-        if (Irp == NULL)
-            continue;
+    IoMarkIrpPending(Irp);
 
-        Pdo->SystemPowerIrp = NULL;
-        KeMemoryBarrier();
+    Exchange = InterlockedExchangePointer(&Pdo->SystemPowerIrp, Irp);
+    ASSERT(Exchange == NULL);
 
-        (VOID) __PdoSetSystemPower(Pdo, Irp);
-    }
+    IoQueueWorkItem(Pdo->SystemPowerWorkItem,
+                    PdoSystemPowerWorker,
+                    DelayedWorkQueue,
+                    Pdo);
 
-    return STATUS_SUCCESS;
+    return STATUS_PENDING;
 }
 
-static DECLSPEC_NOINLINE NTSTATUS
-PdoSetPower(
+static FORCEINLINE NTSTATUS
+__PdoSetPower(
     IN  PXENVIF_PDO     Pdo,
     IN  PIRP            Irp
     )
 {
     PIO_STACK_LOCATION  StackLocation;
     POWER_STATE_TYPE    PowerType;
-    POWER_ACTION        PowerAction;
     NTSTATUS            status;
     
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
     PowerType = StackLocation->Parameters.Power.Type;
-    PowerAction = StackLocation->Parameters.Power.ShutdownType;
-
-    if (PowerAction >= PowerActionShutdown) {
-        Irp->IoStatus.Status = STATUS_SUCCESS;
-
-        status = Irp->IoStatus.Status;
-        IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
-        goto done;
-    }
 
     switch (PowerType) {
     case DevicePowerState:
-        IoMarkIrpPending(Irp);
-
-        ASSERT3P(Pdo->DevicePowerIrp, ==, NULL);
-        Pdo->DevicePowerIrp = Irp;
-        KeMemoryBarrier();
-
-        ThreadWake(Pdo->DevicePowerThread);
-
-        status = STATUS_PENDING;
+        status = __PdoSetDevicePower(Pdo, Irp);
         break;
 
     case SystemPowerState:
-        IoMarkIrpPending(Irp);
-
-        ASSERT3P(Pdo->SystemPowerIrp, ==, NULL);
-        Pdo->SystemPowerIrp = Irp;
-        KeMemoryBarrier();
-
-        ThreadWake(Pdo->SystemPowerThread);
-
-        status = STATUS_PENDING;
+        status = __PdoSetSystemPower(Pdo, Irp);
         break;
 
     default:
@@ -2608,48 +2568,41 @@ PdoSetPower(
         break;
     }
 
-done:    
     return status;
 }
 
-static DECLSPEC_NOINLINE NTSTATUS
-PdoQueryPower(
-    IN  PXENVIF_PDO     Pdo,
-    IN  PIRP            Irp
+static FORCEINLINE NTSTATUS
+__PdoQueryPower(
+    IN  PXENVIF_PDO Pdo,
+    IN  PIRP        Irp
     )
 {
-    NTSTATUS            status;
-    
     UNREFERENCED_PARAMETER(Pdo);
 
     Irp->IoStatus.Status = STATUS_SUCCESS;
-
-    status = Irp->IoStatus.Status;
     IoCompleteRequest(Irp, IO_NO_INCREMENT);
-    
-    return status;
+
+    return STATUS_SUCCESS;
 }
 
-static DECLSPEC_NOINLINE NTSTATUS
+static NTSTATUS
 PdoDispatchPower(
     IN  PXENVIF_PDO     Pdo,
     IN  PIRP            Irp
     )
 {
     PIO_STACK_LOCATION  StackLocation;
-    UCHAR               MinorFunction;
     NTSTATUS            status;
 
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    MinorFunction = StackLocation->MinorFunction;
 
     switch (StackLocation->MinorFunction) {
     case IRP_MN_SET_POWER:
-        status = PdoSetPower(Pdo, Irp);
+        status = __PdoSetPower(Pdo, Irp);
         break;
 
     case IRP_MN_QUERY_POWER:
-        status = PdoQueryPower(Pdo, Irp);
+        status = __PdoQueryPower(Pdo, Irp);
         break;
 
     default:
@@ -2762,12 +2715,12 @@ PdoCreate(
     Pdo->Dx = Dx;
     Pdo->Fdo = Fdo;
 
-    status = ThreadCreate(PdoSystemPower, Pdo, &Pdo->SystemPowerThread);
-    if (!NT_SUCCESS(status))
+    Pdo->SystemPowerWorkItem = IoAllocateWorkItem(PhysicalDeviceObject);
+    if (Pdo->SystemPowerWorkItem == NULL)
         goto fail3;
 
-    status = ThreadCreate(PdoDevicePower, Pdo, &Pdo->DevicePowerThread);
-    if (!NT_SUCCESS(status))
+    Pdo->DevicePowerWorkItem = IoAllocateWorkItem(PhysicalDeviceObject);
+    if (Pdo->DevicePowerWorkItem == NULL)
         goto fail4;
 
     __PdoSetName(Pdo, Number);
@@ -2860,16 +2813,14 @@ fail6:
 fail5:
     Error("fail5\n");
 
-    ThreadAlert(Pdo->DevicePowerThread);
-    ThreadJoin(Pdo->DevicePowerThread);
-    Pdo->DevicePowerThread = NULL;
+    IoFreeWorkItem(Pdo->DevicePowerWorkItem);
+    Pdo->DevicePowerWorkItem = NULL;
 
 fail4:
     Error("fail4\n");
 
-    ThreadAlert(Pdo->SystemPowerThread);
-    ThreadJoin(Pdo->SystemPowerThread);
-    Pdo->SystemPowerThread = NULL;
+    IoFreeWorkItem(Pdo->SystemPowerWorkItem);
+    Pdo->SystemPowerWorkItem = NULL;
 
 fail3:
     Error("fail3\n");
@@ -2939,13 +2890,11 @@ PdoDestroy(
 
     __PdoClearPermanentAddress(Pdo);
 
-    ThreadAlert(Pdo->DevicePowerThread);
-    ThreadJoin(Pdo->DevicePowerThread);
-    Pdo->DevicePowerThread = NULL;
+    IoFreeWorkItem(Pdo->DevicePowerWorkItem);
+    Pdo->DevicePowerWorkItem = NULL;
 
-    ThreadAlert(Pdo->SystemPowerThread);
-    ThreadJoin(Pdo->SystemPowerThread);
-    Pdo->SystemPowerThread = NULL;
+    IoFreeWorkItem(Pdo->SystemPowerWorkItem);
+    Pdo->SystemPowerWorkItem = NULL;
 
     Pdo->Fdo = NULL;
     Pdo->Dx = NULL;
-- 
2.41.0.windows.3



From win-pv-devel-bounces@lists.xenproject.org Tue Mar 12 07:24:11 2024
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Tue, 12 Mar 2024 07:24:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.691649.1077869 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rjwUd-0000Yi-K4; Tue, 12 Mar 2024 07:24:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 691649.1077869; Tue, 12 Mar 2024 07:24:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rjwUd-0000Yb-HC; Tue, 12 Mar 2024 07:24:11 +0000
Received: by outflank-mailman (input) for mailman id 691649;
 Tue, 12 Mar 2024 07:24:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=B9MD=KS=cloud.com=owen.smith@srs-se1.protection.inumbo.net>)
 id 1rjwUc-0000YQ-Qm
 for win-pv-devel@lists.xenproject.org; Tue, 12 Mar 2024 07:24:11 +0000
Received: from mail-lf1-x12c.google.com (mail-lf1-x12c.google.com
 [2a00:1450:4864:20::12c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 82cbf19a-e041-11ee-a1ee-f123f15fe8a2;
 Tue, 12 Mar 2024 08:24:07 +0100 (CET)
Received: by mail-lf1-x12c.google.com with SMTP id
 2adb3069b0e04-5131316693cso7033639e87.0
 for <win-pv-devel@lists.xenproject.org>; Tue, 12 Mar 2024 00:24:06 -0700 (PDT)
Received: from localhost.localdomain
 (host86-169-131-93.range86-169.btcentralplus.com. [86.169.131.93])
 by smtp.gmail.com with ESMTPSA id
 c6-20020a05600c0a4600b00412f2136793sm18332579wmq.44.2024.03.12.00.24.04
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 12 Mar 2024 00:24:04 -0700 (PDT)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 82cbf19a-e041-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1710228245; x=1710833045; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=kJxfEIoOlkmqZ3UzynEZW56VgfbKAFY8rRpToHfWAxA=;
        b=H8Gl7PTBReSbUGhPnHWByReHBhJ7cQLe6kCknrYgEEGmoL9qTl98+LL2dM9/BUGtJY
         FegLH3aOToHO3Rw9OSvGOpXjmrdssHRSM9er/PwkotREIgZZx7E0nV+RnpAjfQLhsCMo
         N7wXd5JNku/68ydM3+o0EqZ5tjL76+Bigl0Xk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1710228245; x=1710833045;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=kJxfEIoOlkmqZ3UzynEZW56VgfbKAFY8rRpToHfWAxA=;
        b=W4rWXUtvu+moHtrKO43oSATDNw/3mDwCvA8pYSIxD+1Dj1K7FFofDCUhQot0SSZFfF
         gNdDKxD6ecSCOOZHeUvtPWx/GTuipISGFCRCAXYnWCVEvuGgbi7DqHbqr5H7JVmtRbEI
         z4Mur07x03FKyYXO80wR4gz6zUttBWbiZTYQkYlEu5ubrUGp9ILMUj5hvCDk6gI2hWIe
         E4o1m4o9RBk3b9uGpZ99r+5yZJOyZeDP4G2b8Qe5kkBcGSoHXPVlWNrYeAOurCZqcoOo
         roK4K9IpjEWSigLnJK5z7JtHlcRQbB1m1BQs54f3V4qa2wzj1tgIloYHYa/vz6nxJ0Bw
         UDQA==
X-Gm-Message-State: AOJu0YxHur3QL2/ZzCHcV+ebsoShajq0nC3zVoD1Vccss1E8L2AKgZn0
	x7I7jdTvM9utz5YwcyZfMjaPtjpewNKrJe4imFfegDxBstB8aGMHmcuNZdu1ftkILOrWDI01zrx
	2qw==
X-Google-Smtp-Source: AGHT+IGwXUOt32PMFx0WKi9xet+8chkTD2BE3gQUQ8WbcwP4/gkjWWiHbn1RXPsEDuC3MMeBD9xJFA==
X-Received: by 2002:ac2:5a05:0:b0:513:af7a:8d82 with SMTP id q5-20020ac25a05000000b00513af7a8d82mr2182407lfn.8.1710228245090;
        Tue, 12 Mar 2024 00:24:05 -0700 (PDT)
From: Owen Smith <owen.smith@cloud.com>
To: win-pv-devel@lists.xenproject.org
Cc: Martin Harvey <Martin.Harvey@citrix.com>,
	Martin Harvey <martin.harvey@citrix.com>,
	Owen Smith <owen.smith@cloud.com>
Subject: [PATCH] Asynchronous power handling.
Date: Tue, 12 Mar 2024 07:23:52 +0000
Message-ID: <20240312072352.1363-1-owen.smith@cloud.com>
X-Mailer: git-send-email 2.41.0.windows.3
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Martin Harvey <Martin.Harvey@citrix.com>

XenDisk requires minimal IRP_MN_SET_POWER/IRP_MN_QUERY_POWER interactions.
No IoWorkItems are required as operations perform no significant work.
Power handlers are is limited to tracking state changes and calling PoSetPowerState.

Signed-off-by: Martin Harvey <martin.harvey@citrix.com>

Refactored
Signed-off-by: Owen Smith <owen.smith@cloud.com>

---
 src/xendisk/fdo.c | 552 +++++++++------------------------------------
 src/xendisk/pdo.c | 555 +++++++++-------------------------------------
 2 files changed, 209 insertions(+), 898 deletions(-)

diff --git a/src/xendisk/fdo.c b/src/xendisk/fdo.c
index 056e24a..2e5c0de 100644
--- a/src/xendisk/fdo.c
+++ b/src/xendisk/fdo.c
@@ -54,11 +54,6 @@ struct _XENDISK_FDO {
     PDEVICE_OBJECT                  LowerDeviceObject;
     PDEVICE_OBJECT                  PhysicalDeviceObject;
 
-    PXENDISK_THREAD                 SystemPowerThread;
-    PIRP                            SystemPowerIrp;
-    PXENDISK_THREAD                 DevicePowerThread;
-    PIRP                            DevicePowerIrp;
-
     MUTEX                           Mutex;
     ULONG                           References;
 };
@@ -1136,36 +1131,50 @@ fail1:
     return status;
 }
 
-static FORCEINLINE NTSTATUS
-__FdoSetDevicePowerUp(
-    IN  PXENDISK_FDO    Fdo,
-    IN  PIRP            Irp
+__drv_functionClass(IO_COMPLETION_ROUTINE)
+__drv_sameIRQL
+static NTSTATUS
+FdoSetDevicePowerUpComplete(
+    IN  PDEVICE_OBJECT  DeviceObject,
+    IN  PIRP            Irp,
+    IN  PVOID           Context
     )
 {
+    PXENDISK_FDO        Fdo = (PXENDISK_FDO) Context;
     PIO_STACK_LOCATION  StackLocation;
-    DEVICE_POWER_STATE  DeviceState;
-    NTSTATUS            status;
+    POWER_STATE         PowerState;
 
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
+    UNREFERENCED_PARAMETER(DeviceObject);
 
-    ASSERT3U(DeviceState, <,  __FdoGetDevicePowerState(Fdo));
+    StackLocation = IoGetCurrentIrpStackLocation(Irp);
+    PowerState = StackLocation->Parameters.Power.State;
 
-    status = FdoForwardIrpSynchronously(Fdo, Irp);
-    if (!NT_SUCCESS(status))
-        goto done;
+    if (Irp->PendingReturned)
+        IoMarkIrpPending(Irp);
 
-    Verbose("%p: %s -> %s\n",
-         Fdo->Dx->DeviceObject,
-         PowerDeviceStateName(__FdoGetDevicePowerState(Fdo)),
-         PowerDeviceStateName(DeviceState));
+    __FdoSetDevicePowerState(Fdo, PowerState.DeviceState);
+    PoSetPowerState(Fdo->Dx->DeviceObject,
+                    DevicePowerState,
+                    PowerState);
 
-    __FdoSetDevicePowerState(Fdo, DeviceState);
+    return STATUS_CONTINUE_COMPLETION;
+}
 
-done:
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
+static FORCEINLINE NTSTATUS
+__FdoSetDevicePowerUp(
+    IN  PXENDISK_FDO    Fdo,
+    IN  PIRP            Irp
+    )
+{
+    IoCopyCurrentIrpStackLocationToNext(Irp);
+    IoSetCompletionRoutine(Irp,
+                           FdoSetDevicePowerUpComplete,
+                           Fdo,
+                           TRUE,
+                           TRUE,
+                           TRUE);
 
-    return status;
+    return IoCallDriver(Fdo->LowerDeviceObject, Irp);
 }
 
 static FORCEINLINE NTSTATUS
@@ -1175,27 +1184,21 @@ __FdoSetDevicePowerDown(
     )
 {
     PIO_STACK_LOCATION  StackLocation;
-    DEVICE_POWER_STATE  DeviceState;
-    NTSTATUS            status;
+    POWER_STATE         PowerState;
 
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
-
-    ASSERT3U(DeviceState, >,  __FdoGetDevicePowerState(Fdo));
-
-    Verbose("%p: %s -> %s\n",
-         Fdo->Dx->DeviceObject,
-         PowerDeviceStateName(__FdoGetDevicePowerState(Fdo)),
-         PowerDeviceStateName(DeviceState));
+    PowerState = StackLocation->Parameters.Power.State;
 
-    __FdoSetDevicePowerState(Fdo, DeviceState);
-
-    status = FdoForwardIrpSynchronously(Fdo, Irp);
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
+    __FdoSetDevicePowerState(Fdo, PowerState.DeviceState);
+    PoSetPowerState(Fdo->Dx->DeviceObject,
+                    DevicePowerState,
+                    PowerState);
 
-    return status;
+    IoSkipCurrentIrpStackLocation(Irp);
+    return IoCallDriver(Fdo->LowerDeviceObject, Irp);
 }
 
+/* IRQL argnostic code, just mark power states.*/
 static FORCEINLINE NTSTATUS
 __FdoSetDevicePower(
     IN  PXENDISK_FDO    Fdo,
@@ -1216,13 +1219,12 @@ __FdoSetDevicePower(
           PowerActionName(PowerAction));
 
     if (DeviceState == __FdoGetDevicePowerState(Fdo)) {
-        status = FdoForwardIrpSynchronously(Fdo, Irp);
-        IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
+        IoSkipCurrentIrpStackLocation(Irp);
+        status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
         goto done;
     }
 
-    status = (DeviceState < __FdoGetDevicePowerState(Fdo)) ?
+    status = DeviceState < __FdoGetDevicePowerState(Fdo) ?
              __FdoSetDevicePowerUp(Fdo, Irp) :
              __FdoSetDevicePowerDown(Fdo, Irp);
 
@@ -1231,39 +1233,51 @@ done:
           PowerDeviceStateName(DeviceState),
           PowerActionName(PowerAction),
           status);
+
     return status;
 }
 
-static FORCEINLINE NTSTATUS
-__FdoSetSystemPowerUp(
-    IN  PXENDISK_FDO    Fdo,
-    IN  PIRP            Irp
+__drv_functionClass(IO_COMPLETION_ROUTINE)
+__drv_sameIRQL
+static NTSTATUS
+FdoSetSystemPowerUpComplete(
+    IN  PDEVICE_OBJECT  DeviceObject,
+    IN  PIRP            Irp,
+    IN  PVOID           Context
     )
 {
+    PXENDISK_FDO        Fdo = (PXENDISK_FDO) Context;
     PIO_STACK_LOCATION  StackLocation;
     SYSTEM_POWER_STATE  SystemState;
-    NTSTATUS            status;
+
+    UNREFERENCED_PARAMETER(DeviceObject);
 
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
     SystemState = StackLocation->Parameters.Power.State.SystemState;
 
-    ASSERT3U(SystemState, <,  __FdoGetSystemPowerState(Fdo));
-
-    status = FdoForwardIrpSynchronously(Fdo, Irp);
-    if (!NT_SUCCESS(status))
-        goto done;
-
-    Verbose("%p: %s -> %s\n",
-         Fdo->Dx->DeviceObject,
-         PowerSystemStateName(__FdoGetSystemPowerState(Fdo)),
-         PowerSystemStateName(SystemState));
+    if (Irp->PendingReturned)
+        IoMarkIrpPending(Irp);
 
     __FdoSetSystemPowerState(Fdo, SystemState);
 
-done:
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
+    return STATUS_CONTINUE_COMPLETION;
+}
 
-    return status;
+static FORCEINLINE NTSTATUS
+__FdoSetSystemPowerUp(
+    IN  PXENDISK_FDO    Fdo,
+    IN  PIRP            Irp
+    )
+{
+    IoCopyCurrentIrpStackLocationToNext(Irp);
+    IoSetCompletionRoutine(Irp,
+                           FdoSetSystemPowerUpComplete,
+                           Fdo,
+                           TRUE,
+                           TRUE,
+                           TRUE);
+
+    return IoCallDriver(Fdo->LowerDeviceObject, Irp);
 }
 
 static FORCEINLINE NTSTATUS
@@ -1274,29 +1288,19 @@ __FdoSetSystemPowerDown(
 {
     PIO_STACK_LOCATION  StackLocation;
     SYSTEM_POWER_STATE  SystemState;
-    NTSTATUS            status;
 
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
     SystemState = StackLocation->Parameters.Power.State.SystemState;
 
-    ASSERT3U(SystemState, >,  __FdoGetSystemPowerState(Fdo));
-
-    Verbose("%p: %s -> %s\n",
-         Fdo->Dx->DeviceObject,
-         PowerSystemStateName(__FdoGetSystemPowerState(Fdo)),
-         PowerSystemStateName(SystemState));
-
     __FdoSetSystemPowerState(Fdo, SystemState);
 
-    status = FdoForwardIrpSynchronously(Fdo, Irp);
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
-    return status;
+    IoSkipCurrentIrpStackLocation(Irp);
+    return IoCallDriver(Fdo->LowerDeviceObject, Irp);
 }
 
 static FORCEINLINE NTSTATUS
 __FdoSetSystemPower(
-    IN  PXENDISK_FDO    Fdo,
+    IN  PXENDISK_FDO     Fdo,
     IN  PIRP            Irp
     )
 {
@@ -1314,13 +1318,12 @@ __FdoSetSystemPower(
           PowerActionName(PowerAction));
 
     if (SystemState == __FdoGetSystemPowerState(Fdo)) {
-        status = FdoForwardIrpSynchronously(Fdo, Irp);
-        IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
+        IoSkipCurrentIrpStackLocation(Irp);
+        status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
         goto done;
     }
 
-    status = (SystemState < __FdoGetSystemPowerState(Fdo)) ?
+    status = SystemState < __FdoGetSystemPowerState(Fdo) ?
              __FdoSetSystemPowerUp(Fdo, Irp) :
              __FdoSetSystemPowerDown(Fdo, Irp);
 
@@ -1329,310 +1332,60 @@ done:
           PowerSystemStateName(SystemState),
           PowerActionName(PowerAction),
           status);
-    return status;
-}
-
-static FORCEINLINE NTSTATUS
-__FdoQueryDevicePowerUp(
-    IN  PXENDISK_FDO    Fdo,
-    IN  PIRP            Irp
-    )
-{
-    PIO_STACK_LOCATION  StackLocation;
-    DEVICE_POWER_STATE  DeviceState;
-    NTSTATUS            status;
-
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
-
-    ASSERT3U(DeviceState, <,  __FdoGetDevicePowerState(Fdo));
-
-    status = FdoForwardIrpSynchronously(Fdo, Irp);
-
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
 
     return status;
 }
 
-static FORCEINLINE NTSTATUS
-__FdoQueryDevicePowerDown(
-    IN  PXENDISK_FDO    Fdo,
-    IN  PIRP            Irp
-    )
-{
-    PIO_STACK_LOCATION  StackLocation;
-    DEVICE_POWER_STATE  DeviceState;
-    NTSTATUS            status;
-
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
-
-    ASSERT3U(DeviceState, >,  __FdoGetDevicePowerState(Fdo));
-
-    status = FdoForwardIrpSynchronously(Fdo, Irp);
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
-    return status;
-}
-
-static FORCEINLINE NTSTATUS
-__FdoQueryDevicePower(
+static NTSTATUS
+FdoDevicePower(
     IN  PXENDISK_FDO    Fdo,
     IN  PIRP            Irp
     )
 {
     PIO_STACK_LOCATION  StackLocation;
-    DEVICE_POWER_STATE  DeviceState;
-    POWER_ACTION        PowerAction;
     NTSTATUS            status;
 
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
-    PowerAction = StackLocation->Parameters.Power.ShutdownType;
 
-    Trace("====> (%s:%s)\n",
-          PowerDeviceStateName(DeviceState),
-          PowerActionName(PowerAction));
-
-    if (DeviceState == __FdoGetDevicePowerState(Fdo)) {
-        status = FdoForwardIrpSynchronously(Fdo, Irp);
-        IoCompleteRequest(Irp, IO_NO_INCREMENT);
+    switch (StackLocation->MinorFunction) {
+    case IRP_MN_SET_POWER:
+        status = __FdoSetDevicePower(Fdo, Irp);
+        break;
 
-        goto done;
+    default:
+        IoSkipCurrentIrpStackLocation(Irp);
+        status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
+        break;
     }
 
-    status = (DeviceState < __FdoGetDevicePowerState(Fdo)) ?
-             __FdoQueryDevicePowerUp(Fdo, Irp) :
-             __FdoQueryDevicePowerDown(Fdo, Irp);
-
-done:
-    Trace("<==== (%s:%s)(%08x)\n",
-          PowerDeviceStateName(DeviceState),
-          PowerActionName(PowerAction),
-          status);
-    return status;
-}
-
-static FORCEINLINE NTSTATUS
-__FdoQuerySystemPowerUp(
-    IN  PXENDISK_FDO     Fdo,
-    IN  PIRP            Irp
-    )
-{
-    PIO_STACK_LOCATION  StackLocation;
-    SYSTEM_POWER_STATE  SystemState;
-    NTSTATUS            status;
-
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    SystemState = StackLocation->Parameters.Power.State.SystemState;
-
-    ASSERT3U(SystemState, <,  __FdoGetSystemPowerState(Fdo));
-
-    status = FdoForwardIrpSynchronously(Fdo, Irp);
-
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
     return status;
 }
 
-static FORCEINLINE NTSTATUS
-__FdoQuerySystemPowerDown(
-    IN  PXENDISK_FDO    Fdo,
-    IN  PIRP            Irp
-    )
-{
-    PIO_STACK_LOCATION  StackLocation;
-    SYSTEM_POWER_STATE  SystemState;
-    NTSTATUS            status;
-
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    SystemState = StackLocation->Parameters.Power.State.SystemState;
-
-    ASSERT3U(SystemState, >,  __FdoGetSystemPowerState(Fdo));
-
-    status = FdoForwardIrpSynchronously(Fdo, Irp);
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
-    return status;
-}
-
-static FORCEINLINE NTSTATUS
-__FdoQuerySystemPower(
+static NTSTATUS
+FdoSystemPower(
     IN  PXENDISK_FDO    Fdo,
     IN  PIRP            Irp
     )
 {
     PIO_STACK_LOCATION  StackLocation;
-    SYSTEM_POWER_STATE  SystemState;
-    POWER_ACTION        PowerAction;
     NTSTATUS            status;
 
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    SystemState = StackLocation->Parameters.Power.State.SystemState;
-    PowerAction = StackLocation->Parameters.Power.ShutdownType;
-
-    Trace("====> (%s:%s)\n",
-          PowerSystemStateName(SystemState),
-          PowerActionName(PowerAction));
 
-    if (SystemState == __FdoGetSystemPowerState(Fdo)) {
-        status = FdoForwardIrpSynchronously(Fdo, Irp);
-        IoCompleteRequest(Irp, IO_NO_INCREMENT);
+    switch (StackLocation->MinorFunction) {
+    case IRP_MN_SET_POWER:
+        status = __FdoSetSystemPower(Fdo, Irp);
+        break;
 
-        goto done;
+    default:
+        IoSkipCurrentIrpStackLocation(Irp);
+        status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
+        break;
     }
 
-    status = (SystemState < __FdoGetSystemPowerState(Fdo)) ?
-             __FdoQuerySystemPowerUp(Fdo, Irp) :
-             __FdoQuerySystemPowerDown(Fdo, Irp);
-
-done:
-    Trace("<==== (%s:%s)(%08x)\n",
-          PowerSystemStateName(SystemState),
-          PowerActionName(PowerAction),
-          status);
-
     return status;
 }
 
-static NTSTATUS
-FdoDevicePower(
-    IN  PXENDISK_THREAD Self,
-    IN  PVOID           Context
-    )
-{
-    PXENDISK_FDO        Fdo = Context;
-    PKEVENT             Event;
-
-    Event = ThreadGetEvent(Self);
-
-    for (;;) {
-        PIRP                Irp;
-        PIO_STACK_LOCATION  StackLocation;
-        UCHAR               MinorFunction;
-
-        if (Fdo->DevicePowerIrp == NULL) {
-            (VOID) KeWaitForSingleObject(Event,
-                                         Executive,
-                                         KernelMode,
-                                         FALSE,
-                                         NULL);
-            KeClearEvent(Event);
-        }
-
-        if (ThreadIsAlerted(Self))
-            break;
-
-        Irp = Fdo->DevicePowerIrp;
-
-        if (Irp == NULL)
-            continue;
-
-        Fdo->DevicePowerIrp = NULL;
-        KeMemoryBarrier();
-
-        StackLocation = IoGetCurrentIrpStackLocation(Irp);
-        MinorFunction = StackLocation->MinorFunction;
-
-        switch (StackLocation->MinorFunction) {
-        case IRP_MN_SET_POWER:
-            (VOID) __FdoSetDevicePower(Fdo, Irp);
-            break;
-
-        case IRP_MN_QUERY_POWER:
-            (VOID) __FdoQueryDevicePower(Fdo, Irp);
-            break;
-
-        default:
-            ASSERT(FALSE);
-            break;
-        }
-
-        IoReleaseRemoveLock(&Fdo->Dx->RemoveLock, Irp);
-    }
-
-    return STATUS_SUCCESS;
-}
-
-static NTSTATUS
-FdoSystemPower(
-    IN  PXENDISK_THREAD Self,
-    IN  PVOID           Context
-    )
-{
-    PXENDISK_FDO        Fdo = Context;
-    PKEVENT             Event;
-
-    Event = ThreadGetEvent(Self);
-
-    for (;;) {
-        PIRP                Irp;
-        PIO_STACK_LOCATION  StackLocation;
-        UCHAR               MinorFunction;
-
-        if (Fdo->SystemPowerIrp == NULL) {
-            (VOID) KeWaitForSingleObject(Event,
-                                         Executive,
-                                         KernelMode,
-                                         FALSE,
-                                         NULL);
-            KeClearEvent(Event);
-        }
-
-        if (ThreadIsAlerted(Self))
-            break;
-
-        Irp = Fdo->SystemPowerIrp;
-
-        if (Irp == NULL)
-            continue;
-
-        Fdo->SystemPowerIrp = NULL;
-        KeMemoryBarrier();
-
-        StackLocation = IoGetCurrentIrpStackLocation(Irp);
-        MinorFunction = StackLocation->MinorFunction;
-
-        switch (StackLocation->MinorFunction) {
-        case IRP_MN_SET_POWER:
-            (VOID) __FdoSetSystemPower(Fdo, Irp);
-            break;
-
-        case IRP_MN_QUERY_POWER:
-            (VOID) __FdoQuerySystemPower(Fdo, Irp);
-            break;
-
-        default:
-            ASSERT(FALSE);
-            break;
-        }
-
-        IoReleaseRemoveLock(&Fdo->Dx->RemoveLock, Irp);
-    }
-
-    return STATUS_SUCCESS;
-}
-
-__drv_functionClass(IO_COMPLETION_ROUTINE)
-__drv_sameIRQL
-static NTSTATUS
-__FdoDispatchPower(
-    IN  PDEVICE_OBJECT  DeviceObject,
-    IN  PIRP            Irp,
-    IN  PVOID           Context
-    )
-{
-    PXENDISK_FDO        Fdo = Context;
-
-    UNREFERENCED_PARAMETER(DeviceObject);
-
-    if (Irp->PendingReturned)
-        IoMarkIrpPending(Irp);
-
-    IoReleaseRemoveLock(&Fdo->Dx->RemoveLock, Irp);
-    return STATUS_SUCCESS;
-}
-
 static DECLSPEC_NOINLINE NTSTATUS
 FdoDispatchPower(
     IN  PXENDISK_FDO    Fdo,
@@ -1640,90 +1393,27 @@ FdoDispatchPower(
     )
 {
     PIO_STACK_LOCATION  StackLocation;
-    UCHAR               MinorFunction;
     POWER_STATE_TYPE    PowerType;
     NTSTATUS            status;
 
-    status = IoAcquireRemoveLock(&Fdo->Dx->RemoveLock, Irp);
-    if (!NT_SUCCESS(status))
-        goto fail1;
-
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    MinorFunction = StackLocation->MinorFunction;
-
-    if (MinorFunction != IRP_MN_QUERY_POWER &&
-        MinorFunction != IRP_MN_SET_POWER) {
-        IoCopyCurrentIrpStackLocationToNext(Irp);
-        IoSetCompletionRoutine(Irp,
-                               __FdoDispatchPower,
-                               Fdo,
-                               TRUE,
-                               TRUE,
-                               TRUE);
-
-        status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
-
-        goto done;
-    }
-
     PowerType = StackLocation->Parameters.Power.Type;
 
-    Trace("====> (%02x:%s)\n",
-          MinorFunction,
-          PowerMinorFunctionName(MinorFunction));
-
     switch (PowerType) {
     case DevicePowerState:
-        IoMarkIrpPending(Irp);
-
-        ASSERT3P(Fdo->DevicePowerIrp, ==, NULL);
-        Fdo->DevicePowerIrp = Irp;
-        KeMemoryBarrier();
-
-        ThreadWake(Fdo->DevicePowerThread);
-
-        status = STATUS_PENDING;
+        status = FdoDevicePower(Fdo, Irp);
         break;
 
     case SystemPowerState:
-        IoMarkIrpPending(Irp);
-
-        ASSERT3P(Fdo->SystemPowerIrp, ==, NULL);
-        Fdo->SystemPowerIrp = Irp;
-        KeMemoryBarrier();
-
-        ThreadWake(Fdo->SystemPowerThread);
-
-        status = STATUS_PENDING;
+        status = FdoSystemPower(Fdo, Irp);
         break;
 
     default:
-        IoCopyCurrentIrpStackLocationToNext(Irp);
-        IoSetCompletionRoutine(Irp,
-                               __FdoDispatchPower,
-                               Fdo,
-                               TRUE,
-                               TRUE,
-                               TRUE);
-
+        IoSkipCurrentIrpStackLocation(Irp);
         status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
         break;
     }
 
-    Trace("<==== (%02x:%s) (%08x)\n",
-          MinorFunction,
-          PowerMinorFunctionName(MinorFunction),
-          status);
-
-done:
-    return status;
-
-fail1:
-    Error("fail1 (%08x)\n", status);
-
-    Irp->IoStatus.Status = status;
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
     return status;
 }
 
@@ -1864,14 +1554,6 @@ FdoCreate(
     Fdo->PhysicalDeviceObject = PhysicalDeviceObject;
     Fdo->LowerDeviceObject = LowerDeviceObject;
 
-    status = ThreadCreate(FdoSystemPower, Fdo, &Fdo->SystemPowerThread);
-    if (!NT_SUCCESS(status))
-        goto fail4;
-
-    status = ThreadCreate(FdoDevicePower, Fdo, &Fdo->DevicePowerThread);
-    if (!NT_SUCCESS(status))
-        goto fail5;
-
     InitializeMutex(&Fdo->Mutex);
     InitializeListHead(&Dx->ListEntry);
     Fdo->References = 1;
@@ -1889,22 +1571,6 @@ FdoCreate(
 
     return STATUS_SUCCESS;
 
-fail5:
-    Error("fail5\n");
-
-    ThreadAlert(Fdo->SystemPowerThread);
-    ThreadJoin(Fdo->SystemPowerThread);
-    Fdo->SystemPowerThread = NULL;
-
-fail4:
-    Error("fail4\n");
-
-    Fdo->PhysicalDeviceObject = NULL;
-    Fdo->LowerDeviceObject = NULL;
-    Fdo->Dx = NULL;
-
-    IoDetachDevice(LowerDeviceObject);
-
 fail3:
     Error("fail3\n");
 
@@ -1939,14 +1605,6 @@ FdoDestroy(
 
     RtlZeroMemory(&Fdo->Mutex, sizeof (MUTEX));
 
-    ThreadAlert(Fdo->DevicePowerThread);
-    ThreadJoin(Fdo->DevicePowerThread);
-    Fdo->DevicePowerThread = NULL;
-
-    ThreadAlert(Fdo->SystemPowerThread);
-    ThreadJoin(Fdo->SystemPowerThread);
-    Fdo->SystemPowerThread = NULL;
-
     Fdo->LowerDeviceObject = NULL;
     Fdo->PhysicalDeviceObject = NULL;
     Fdo->Dx = NULL;
diff --git a/src/xendisk/pdo.c b/src/xendisk/pdo.c
index 321ecf3..f714efb 100644
--- a/src/xendisk/pdo.c
+++ b/src/xendisk/pdo.c
@@ -60,11 +60,6 @@ struct _XENDISK_PDO {
     PDEVICE_OBJECT              PhysicalDeviceObject;
     CHAR                        Name[MAXNAMELEN];
 
-    PXENDISK_THREAD             SystemPowerThread;
-    PIRP                        SystemPowerIrp;
-    PXENDISK_THREAD             DevicePowerThread;
-    PIRP                        DevicePowerIrp;
-
     PXENDISK_FDO                Fdo;
 
     BOOLEAN                     InterceptTrim;
@@ -1416,37 +1411,50 @@ fail1:
     return status;
 }
 
-static FORCEINLINE NTSTATUS
-__PdoSetDevicePowerUp(
-    IN  PXENDISK_PDO    Pdo,
-    IN  PIRP            Irp
+__drv_functionClass(IO_COMPLETION_ROUTINE)
+__drv_sameIRQL
+static NTSTATUS
+PdoSetDevicePowerUpComplete(
+    IN  PDEVICE_OBJECT  DeviceObject,
+    IN  PIRP            Irp,
+    IN  PVOID           Context
     )
 {
+    PXENDISK_PDO        Pdo = (PXENDISK_PDO) Context;
     PIO_STACK_LOCATION  StackLocation;
-    DEVICE_POWER_STATE  DeviceState;
-    NTSTATUS            status;
+    POWER_STATE         PowerState;
 
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
+    UNREFERENCED_PARAMETER(DeviceObject);
 
-    ASSERT3U(DeviceState, <,  __PdoGetDevicePowerState(Pdo));
+    StackLocation = IoGetCurrentIrpStackLocation(Irp);
+    PowerState = StackLocation->Parameters.Power.State;
 
-    status = PdoForwardIrpSynchronously(Pdo, Irp);
-    if (!NT_SUCCESS(status))
-        goto done;
+    if (Irp->PendingReturned)
+        IoMarkIrpPending(Irp);
 
-    Verbose("%s: %s -> %s\n",
-            __PdoGetName(Pdo),
-            PowerDeviceStateName(__PdoGetDevicePowerState(Pdo)),
-            PowerDeviceStateName(DeviceState));
+    __PdoSetDevicePowerState(Pdo, PowerState.DeviceState);
+    PoSetPowerState(Pdo->Dx->DeviceObject,
+                    DevicePowerState,
+                    PowerState);
 
-    __PdoSetDevicePowerState(Pdo, DeviceState);
+    return STATUS_CONTINUE_COMPLETION;
+}
 
-done:
-    Irp->IoStatus.Status = status;
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
+static FORCEINLINE NTSTATUS
+__PdoSetDevicePowerUp(
+    IN  PXENDISK_PDO    Pdo,
+    IN  PIRP            Irp
+    )
+{
+    IoCopyCurrentIrpStackLocationToNext(Irp);
+    IoSetCompletionRoutine(Irp,
+                           PdoSetDevicePowerUpComplete,
+                           Pdo,
+                           TRUE,
+                           TRUE,
+                           TRUE);
 
-    return status;
+    return IoCallDriver(Pdo->LowerDeviceObject, Irp);
 }
 
 static FORCEINLINE NTSTATUS
@@ -1456,25 +1464,18 @@ __PdoSetDevicePowerDown(
     )
 {
     PIO_STACK_LOCATION  StackLocation;
-    DEVICE_POWER_STATE  DeviceState;
-    NTSTATUS            status;
+    POWER_STATE         PowerState;
 
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
-
-    ASSERT3U(DeviceState, >,  __PdoGetDevicePowerState(Pdo));
-
-    Verbose("%s: %s -> %s\n",
-            __PdoGetName(Pdo),
-            PowerDeviceStateName(__PdoGetDevicePowerState(Pdo)),
-            PowerDeviceStateName(DeviceState));
+    PowerState = StackLocation->Parameters.Power.State;
 
-    __PdoSetDevicePowerState(Pdo, DeviceState);
-
-    status = PdoForwardIrpSynchronously(Pdo, Irp);
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
+    __PdoSetDevicePowerState(Pdo, PowerState.DeviceState);
+    PoSetPowerState(Pdo->Dx->DeviceObject,
+                    DevicePowerState,
+                    PowerState);
 
-    return status;
+    IoSkipCurrentIrpStackLocation(Irp);
+    return IoCallDriver(Pdo->LowerDeviceObject, Irp);
 }
 
 static FORCEINLINE NTSTATUS
@@ -1497,13 +1498,12 @@ __PdoSetDevicePower(
           PowerActionName(PowerAction));
 
     if (DeviceState == __PdoGetDevicePowerState(Pdo)) {
-        status = PdoForwardIrpSynchronously(Pdo, Irp);
-        IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
+        IoSkipCurrentIrpStackLocation(Irp);
+        status = IoCallDriver(Pdo->LowerDeviceObject, Irp);
         goto done;
     }
 
-    status = (DeviceState < __PdoGetDevicePowerState(Pdo)) ?
+    status = DeviceState < __PdoGetDevicePowerState(Pdo) ?
              __PdoSetDevicePowerUp(Pdo, Irp) :
              __PdoSetDevicePowerDown(Pdo, Irp);
 
@@ -1512,40 +1512,51 @@ done:
           PowerDeviceStateName(DeviceState),
           PowerActionName(PowerAction),
           status);
+
     return status;
 }
 
-static FORCEINLINE NTSTATUS
-__PdoSetSystemPowerUp(
-    IN  PXENDISK_PDO    Pdo,
-    IN  PIRP            Irp
+__drv_functionClass(IO_COMPLETION_ROUTINE)
+__drv_sameIRQL
+static NTSTATUS
+PdoSetSystemPowerUpComplete(
+    IN  PDEVICE_OBJECT  DeviceObject,
+    IN  PIRP            Irp,
+    IN  PVOID           Context
     )
 {
+    PXENDISK_PDO        Pdo = (PXENDISK_PDO) Context;
     PIO_STACK_LOCATION  StackLocation;
     SYSTEM_POWER_STATE  SystemState;
-    NTSTATUS            status;
+
+    UNREFERENCED_PARAMETER(DeviceObject);
 
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
     SystemState = StackLocation->Parameters.Power.State.SystemState;
 
-    ASSERT3U(SystemState, <,  __PdoGetSystemPowerState(Pdo));
-
-    status = PdoForwardIrpSynchronously(Pdo, Irp);
-    if (!NT_SUCCESS(status))
-        goto done;
-
-    Verbose("%s: %s -> %s\n",
-            __PdoGetName(Pdo),
-            PowerSystemStateName(__PdoGetSystemPowerState(Pdo)),
-            PowerSystemStateName(SystemState));
+    if (Irp->PendingReturned)
+        IoMarkIrpPending(Irp);
 
     __PdoSetSystemPowerState(Pdo, SystemState);
 
-done:
-    Irp->IoStatus.Status = status;
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
+    return STATUS_CONTINUE_COMPLETION;
+}
 
-    return status;
+static FORCEINLINE NTSTATUS
+__PdoSetSystemPowerUp(
+    IN  PXENDISK_PDO    Pdo,
+    IN  PIRP            Irp
+    )
+{
+    IoCopyCurrentIrpStackLocationToNext(Irp);
+    IoSetCompletionRoutine(Irp,
+                           PdoSetSystemPowerUpComplete,
+                           Pdo,
+                           TRUE,
+                           TRUE,
+                           TRUE);
+
+    return IoCallDriver(Pdo->LowerDeviceObject, Irp);
 }
 
 static FORCEINLINE NTSTATUS
@@ -1556,24 +1567,14 @@ __PdoSetSystemPowerDown(
 {
     PIO_STACK_LOCATION  StackLocation;
     SYSTEM_POWER_STATE  SystemState;
-    NTSTATUS            status;
 
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
     SystemState = StackLocation->Parameters.Power.State.SystemState;
 
-    ASSERT3U(SystemState, >,  __PdoGetSystemPowerState(Pdo));
-
-    Verbose("%s: %s -> %s\n",
-            __PdoGetName(Pdo),
-            PowerSystemStateName(__PdoGetSystemPowerState(Pdo)),
-            PowerSystemStateName(SystemState));
-
     __PdoSetSystemPowerState(Pdo, SystemState);
 
-    status = PdoForwardIrpSynchronously(Pdo, Irp);
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
-    return status;
+    IoSkipCurrentIrpStackLocation(Irp);
+    return IoCallDriver(Pdo->LowerDeviceObject, Irp);
 }
 
 static FORCEINLINE NTSTATUS
@@ -1596,13 +1597,12 @@ __PdoSetSystemPower(
           PowerActionName(PowerAction));
 
     if (SystemState == __PdoGetSystemPowerState(Pdo)) {
-        status = PdoForwardIrpSynchronously(Pdo, Irp);
-        IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
+        IoSkipCurrentIrpStackLocation(Irp);
+        status = IoCallDriver(Pdo->LowerDeviceObject, Irp);
         goto done;
     }
 
-    status = (SystemState < __PdoGetSystemPowerState(Pdo)) ?
+    status = SystemState < __PdoGetSystemPowerState(Pdo) ?
              __PdoSetSystemPowerUp(Pdo, Irp) :
              __PdoSetSystemPowerDown(Pdo, Irp);
 
@@ -1611,403 +1611,88 @@ done:
           PowerSystemStateName(SystemState),
           PowerActionName(PowerAction),
           status);
-    return status;
-}
-
-static FORCEINLINE NTSTATUS
-__PdoQueryDevicePowerUp(
-    IN  PXENDISK_PDO    Pdo,
-    IN  PIRP            Irp
-    )
-{
-    PIO_STACK_LOCATION  StackLocation;
-    DEVICE_POWER_STATE  DeviceState;
-    NTSTATUS            status;
-
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
-
-    ASSERT3U(DeviceState, <,  __PdoGetDevicePowerState(Pdo));
-
-    status = PdoForwardIrpSynchronously(Pdo, Irp);
-
-    Irp->IoStatus.Status = status;
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
 
     return status;
 }
 
-static FORCEINLINE NTSTATUS
-__PdoQueryDevicePowerDown(
+static NTSTATUS
+PdoDevicePower(
     IN  PXENDISK_PDO    Pdo,
     IN  PIRP            Irp
     )
 {
-    PIO_STACK_LOCATION  StackLocation;
-    DEVICE_POWER_STATE  DeviceState;
     NTSTATUS            status;
-
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
-
-    ASSERT3U(DeviceState, >,  __PdoGetDevicePowerState(Pdo));
-
-    status = PdoForwardIrpSynchronously(Pdo, Irp);
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
-    return status;
-}
-
-static FORCEINLINE NTSTATUS
-__PdoQueryDevicePower(
-    IN  PXENDISK_PDO    Pdo,
-    IN  PIRP            Irp
-    )
-{
     PIO_STACK_LOCATION  StackLocation;
-    DEVICE_POWER_STATE  DeviceState;
-    POWER_ACTION        PowerAction;
-    NTSTATUS            status;
 
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
-    PowerAction = StackLocation->Parameters.Power.ShutdownType;
 
-    Trace("====> (%s:%s)\n",
-          PowerDeviceStateName(DeviceState),
-          PowerActionName(PowerAction));
-
-    if (DeviceState == __PdoGetDevicePowerState(Pdo)) {
-        status = PdoForwardIrpSynchronously(Pdo, Irp);
-        IoCompleteRequest(Irp, IO_NO_INCREMENT);
+    switch (StackLocation->MinorFunction) {
+    case IRP_MN_SET_POWER:
+        status = __PdoSetDevicePower(Pdo, Irp);
+        break;
 
-        goto done;
+    default:
+        IoSkipCurrentIrpStackLocation(Irp);
+        status = IoCallDriver(Pdo->LowerDeviceObject, Irp);
+        break;
     }
 
-    status = (DeviceState < __PdoGetDevicePowerState(Pdo)) ?
-             __PdoQueryDevicePowerUp(Pdo, Irp) :
-             __PdoQueryDevicePowerDown(Pdo, Irp);
-
-done:
-    Trace("<==== (%s:%s)(%08x)\n",
-          PowerDeviceStateName(DeviceState),
-          PowerActionName(PowerAction),
-          status);
-    return status;
-}
-
-static FORCEINLINE NTSTATUS
-__PdoQuerySystemPowerUp(
-    IN  PXENDISK_PDO     Pdo,
-    IN  PIRP            Irp
-    )
-{
-    PIO_STACK_LOCATION  StackLocation;
-    SYSTEM_POWER_STATE  SystemState;
-    NTSTATUS            status;
-
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    SystemState = StackLocation->Parameters.Power.State.SystemState;
-
-    ASSERT3U(SystemState, <,  __PdoGetSystemPowerState(Pdo));
-
-    status = PdoForwardIrpSynchronously(Pdo, Irp);
-
-    Irp->IoStatus.Status = status;
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
     return status;
 }
 
-static FORCEINLINE NTSTATUS
-__PdoQuerySystemPowerDown(
+static NTSTATUS
+PdoSystemPower(
     IN  PXENDISK_PDO    Pdo,
     IN  PIRP            Irp
     )
 {
-    PIO_STACK_LOCATION  StackLocation;
-    SYSTEM_POWER_STATE  SystemState;
     NTSTATUS            status;
-
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    SystemState = StackLocation->Parameters.Power.State.SystemState;
-
-    ASSERT3U(SystemState, >,  __PdoGetSystemPowerState(Pdo));
-
-    status = PdoForwardIrpSynchronously(Pdo, Irp);
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
-    return status;
-}
-
-static FORCEINLINE NTSTATUS
-__PdoQuerySystemPower(
-    IN  PXENDISK_PDO    Pdo,
-    IN  PIRP            Irp
-    )
-{
     PIO_STACK_LOCATION  StackLocation;
-    SYSTEM_POWER_STATE  SystemState;
-    POWER_ACTION        PowerAction;
-    NTSTATUS            status;
 
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    SystemState = StackLocation->Parameters.Power.State.SystemState;
-    PowerAction = StackLocation->Parameters.Power.ShutdownType;
 
-    Trace("====> (%s:%s)\n",
-          PowerSystemStateName(SystemState),
-          PowerActionName(PowerAction));
-
-    if (SystemState == __PdoGetSystemPowerState(Pdo)) {
-        status = PdoForwardIrpSynchronously(Pdo, Irp);
-        IoCompleteRequest(Irp, IO_NO_INCREMENT);
+    switch (StackLocation->MinorFunction) {
+    case IRP_MN_SET_POWER:
+        status = __PdoSetSystemPower(Pdo, Irp);
+        break;
 
-        goto done;
+    default:
+        IoSkipCurrentIrpStackLocation(Irp);
+        status = IoCallDriver(Pdo->LowerDeviceObject, Irp);
+        break;
     }
 
-    status = (SystemState < __PdoGetSystemPowerState(Pdo)) ?
-             __PdoQuerySystemPowerUp(Pdo, Irp) :
-             __PdoQuerySystemPowerDown(Pdo, Irp);
-
-done:
-    Trace("<==== (%s:%s)(%08x)\n",
-          PowerSystemStateName(SystemState),
-          PowerActionName(PowerAction),
-          status);
-
     return status;
 }
 
-static NTSTATUS
-PdoDevicePower(
-    IN  PXENDISK_THREAD Self,
-    IN  PVOID           Context
-    )
-{
-    PXENDISK_PDO        Pdo = Context;
-    PKEVENT             Event;
-
-    Event = ThreadGetEvent(Self);
-
-    for (;;) {
-        PIRP                Irp;
-        PIO_STACK_LOCATION  StackLocation;
-        UCHAR               MinorFunction;
-
-        if (Pdo->DevicePowerIrp == NULL) {
-            (VOID) KeWaitForSingleObject(Event,
-                                         Executive,
-                                         KernelMode,
-                                         FALSE,
-                                         NULL);
-            KeClearEvent(Event);
-        }
-
-        if (ThreadIsAlerted(Self))
-            break;
-
-        Irp = Pdo->DevicePowerIrp;
-
-        if (Irp == NULL)
-            continue;
-
-        Pdo->DevicePowerIrp = NULL;
-        KeMemoryBarrier();
-
-        StackLocation = IoGetCurrentIrpStackLocation(Irp);
-        MinorFunction = StackLocation->MinorFunction;
-
-        switch (StackLocation->MinorFunction) {
-        case IRP_MN_SET_POWER:
-            (VOID) __PdoSetDevicePower(Pdo, Irp);
-            break;
-
-        case IRP_MN_QUERY_POWER:
-            (VOID) __PdoQueryDevicePower(Pdo, Irp);
-            break;
-
-        default:
-            ASSERT(FALSE);
-            break;
-        }
-
-        IoReleaseRemoveLock(&Pdo->Dx->RemoveLock, Irp);
-    }
-
-    return STATUS_SUCCESS;
-}
-
-static NTSTATUS
-PdoSystemPower(
-    IN  PXENDISK_THREAD Self,
-    IN  PVOID           Context
-    )
-{
-    PXENDISK_PDO        Pdo = Context;
-    PKEVENT             Event;
-
-    Event = ThreadGetEvent(Self);
-
-    for (;;) {
-        PIRP                Irp;
-        PIO_STACK_LOCATION  StackLocation;
-        UCHAR               MinorFunction;
-
-        if (Pdo->SystemPowerIrp == NULL) {
-            (VOID) KeWaitForSingleObject(Event,
-                                         Executive,
-                                         KernelMode,
-                                         FALSE,
-                                         NULL);
-            KeClearEvent(Event);
-        }
-
-        if (ThreadIsAlerted(Self))
-            break;
-
-        Irp = Pdo->SystemPowerIrp;
-
-        if (Irp == NULL)
-            continue;
-
-        Pdo->SystemPowerIrp = NULL;
-        KeMemoryBarrier();
-
-        StackLocation = IoGetCurrentIrpStackLocation(Irp);
-        MinorFunction = StackLocation->MinorFunction;
-
-        switch (StackLocation->MinorFunction) {
-        case IRP_MN_SET_POWER:
-            (VOID) __PdoSetSystemPower(Pdo, Irp);
-            break;
-
-        case IRP_MN_QUERY_POWER:
-            (VOID) __PdoQuerySystemPower(Pdo, Irp);
-            break;
-
-        default:
-            ASSERT(FALSE);
-            break;
-        }
-
-        IoReleaseRemoveLock(&Pdo->Dx->RemoveLock, Irp);
-    }
-
-    return STATUS_SUCCESS;
-}
-
-__drv_functionClass(IO_COMPLETION_ROUTINE)
-__drv_sameIRQL
-static NTSTATUS
-__PdoDispatchPower(
-    IN  PDEVICE_OBJECT  DeviceObject,
-    IN  PIRP            Irp,
-    IN  PVOID           Context
-    )
-{
-    PXENDISK_PDO        Pdo = Context;
-
-    UNREFERENCED_PARAMETER(DeviceObject);
-
-    if (Irp->PendingReturned)
-        IoMarkIrpPending(Irp);
-
-    IoReleaseRemoveLock(&Pdo->Dx->RemoveLock, Irp);
-    return STATUS_SUCCESS;
-}
-
 static DECLSPEC_NOINLINE NTSTATUS
 PdoDispatchPower(
-    IN  PXENDISK_PDO    Pdo,
+    IN  PXENDISK_PDO   Pdo,
     IN  PIRP            Irp
     )
 {
     PIO_STACK_LOCATION  StackLocation;
-    UCHAR               MinorFunction;
     POWER_STATE_TYPE    PowerType;
     NTSTATUS            status;
 
-    status = IoAcquireRemoveLock(&Pdo->Dx->RemoveLock, Irp);
-    if (!NT_SUCCESS(status))
-        goto fail1;
-
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    MinorFunction = StackLocation->MinorFunction;
-
-    if (MinorFunction != IRP_MN_QUERY_POWER &&
-        MinorFunction != IRP_MN_SET_POWER) {
-        IoCopyCurrentIrpStackLocationToNext(Irp);
-        IoSetCompletionRoutine(Irp,
-                               __PdoDispatchPower,
-                               Pdo,
-                               TRUE,
-                               TRUE,
-                               TRUE);
-
-        status = IoCallDriver(Pdo->LowerDeviceObject, Irp);
-
-        goto done;
-    }
-
     PowerType = StackLocation->Parameters.Power.Type;
 
-    Trace("====> (%02x:%s)\n",
-          MinorFunction,
-          PowerMinorFunctionName(MinorFunction));
-
     switch (PowerType) {
     case DevicePowerState:
-        IoMarkIrpPending(Irp);
-
-        ASSERT3P(Pdo->DevicePowerIrp, ==, NULL);
-        Pdo->DevicePowerIrp = Irp;
-        KeMemoryBarrier();
-
-        ThreadWake(Pdo->DevicePowerThread);
-
-        status = STATUS_PENDING;
+        status = PdoDevicePower(Pdo, Irp);
         break;
 
     case SystemPowerState:
-        IoMarkIrpPending(Irp);
-
-        ASSERT3P(Pdo->SystemPowerIrp, ==, NULL);
-        Pdo->SystemPowerIrp = Irp;
-        KeMemoryBarrier();
-
-        ThreadWake(Pdo->SystemPowerThread);
-
-        status = STATUS_PENDING;
+        status = PdoSystemPower(Pdo, Irp);
         break;
 
     default:
-        IoCopyCurrentIrpStackLocationToNext(Irp);
-        IoSetCompletionRoutine(Irp,
-                               __PdoDispatchPower,
-                               Pdo,
-                               TRUE,
-                               TRUE,
-                               TRUE);
-
+        IoSkipCurrentIrpStackLocation(Irp);
         status = IoCallDriver(Pdo->LowerDeviceObject, Irp);
         break;
     }
 
-    Trace("<==== (%02x:%s) (%08x)\n",
-          MinorFunction,
-          PowerMinorFunctionName(MinorFunction),
-          status);
-
-done:
-    return status;
-
-fail1:
-    Error("fail1 (%08x)\n", status);
-
-    Irp->IoStatus.Status = status;
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
     return status;
 }
 
@@ -2157,14 +1842,6 @@ PdoCreate(
     Pdo->PhysicalDeviceObject = PhysicalDeviceObject;
     Pdo->LowerDeviceObject = LowerDeviceObject;
 
-    status = ThreadCreate(PdoSystemPower, Pdo, &Pdo->SystemPowerThread);
-    if (!NT_SUCCESS(status))
-        goto fail4;
-
-    status = ThreadCreate(PdoDevicePower, Pdo, &Pdo->DevicePowerThread);
-    if (!NT_SUCCESS(status))
-        goto fail5;
-
     __PdoSetName(Pdo, DeviceID, InstanceID);
 
     ParametersKey = DriverGetParametersKey();
@@ -2192,22 +1869,6 @@ PdoCreate(
 
     return STATUS_SUCCESS;
 
-fail5:
-    Error("fail5\n");
-
-    ThreadAlert(Pdo->SystemPowerThread);
-    ThreadJoin(Pdo->SystemPowerThread);
-    Pdo->SystemPowerThread = NULL;
-
-fail4:
-    Error("fail4\n");
-
-    Pdo->PhysicalDeviceObject = NULL;
-    Pdo->LowerDeviceObject = NULL;
-    Pdo->Dx = NULL;
-
-    IoDetachDevice(LowerDeviceObject);
-
 fail3:
     Error("fail3\n");
 
@@ -2246,14 +1907,6 @@ PdoDestroy(
 
     RtlZeroMemory(Pdo->Name, sizeof (Pdo->Name));
 
-    ThreadAlert(Pdo->DevicePowerThread);
-    ThreadJoin(Pdo->DevicePowerThread);
-    Pdo->DevicePowerThread = NULL;
-
-    ThreadAlert(Pdo->SystemPowerThread);
-    ThreadJoin(Pdo->SystemPowerThread);
-    Pdo->SystemPowerThread = NULL;
-
     Pdo->PhysicalDeviceObject = NULL;
     Pdo->LowerDeviceObject = NULL;
     Pdo->Dx = NULL;
-- 
2.41.0.windows.3



From win-pv-devel-bounces@lists.xenproject.org Wed Mar 13 08:14:18 2024
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Wed, 13 Mar 2024 08:14:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.692297.1079266 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rkJkc-0002Wg-JT; Wed, 13 Mar 2024 08:14:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 692297.1079266; Wed, 13 Mar 2024 08:14:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rkJkc-0002Wb-Gm; Wed, 13 Mar 2024 08:14:14 +0000
Received: by outflank-mailman (input) for mailman id 692297;
 Wed, 13 Mar 2024 08:14:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DzzD=KT=cloud.com=owen.smith@srs-se1.protection.inumbo.net>)
 id 1rkJkb-0002WV-OY
 for win-pv-devel@lists.xenproject.org; Wed, 13 Mar 2024 08:14:13 +0000
Received: from mail-lj1-x229.google.com (mail-lj1-x229.google.com
 [2a00:1450:4864:20::229])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ab1da101-e111-11ee-a1ee-f123f15fe8a2;
 Wed, 13 Mar 2024 09:14:10 +0100 (CET)
Received: by mail-lj1-x229.google.com with SMTP id
 38308e7fff4ca-2d2505352e6so9488521fa.3
 for <win-pv-devel@lists.xenproject.org>; Wed, 13 Mar 2024 01:14:09 -0700 (PDT)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ab1da101-e111-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1710317649; x=1710922449; darn=lists.xenproject.org;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:from:to:cc:subject:date:message-id:reply-to;
        bh=n6XiWlB42A+9AnDgjQxrfniqMMTzAicCl4tbgbDmD0I=;
        b=j+8E/aOVqwBCGxsvbZoHaceaLCEzH90h0DyUp6yi6GI++LYarJoxMstUaLkR8DwE/T
         L/wzxSbBgrAr/tLPb9LO2VqwNsRLtxXT6xd6EuGNtBnp+U1YPk78wzObAqAWrKhXxOHN
         P23BY0Qk+NcUw2s8u+7/Z9CsFv4K8jZjp83hY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1710317649; x=1710922449;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=n6XiWlB42A+9AnDgjQxrfniqMMTzAicCl4tbgbDmD0I=;
        b=iNZ5uETPOSdZqQ2EIapypGH8AtWDh+4W6jSSas9DIZTi7mNydGcNVO4xkp40HYpsYy
         V4ivKpWGYgCLzytcMPS6hZjDlCvfksTifUy0xmnRxN1K2EBBxEibl7uI8jCaPIACOq9x
         8pKk9R60SVVoEFf48p5ei/Q1alfykm0jsBy8RomL656bGueoxSF/SzTGd3cQYEtfrDaD
         VwTTo0uMrOIrbA+D4C6fKQHP/pPHIe/fOMcE4X3uknRAoudvjg1oNGiS9UHiZWxOl4hd
         d1IM5UrEVC+6v6y5z17Gm70i+TwpRB+/zsbtk/IPeocWpt9lVdWGxAkbBV9ijaev1Szu
         UqjA==
X-Gm-Message-State: AOJu0YwOzSW7m+thOOYr5cWLZHlQWcbcUjS1qI7vj/xzbFSllEjlhh0V
	CEyf2YAj7CKpSETiplkSQbZSxjxnUdecs2S1kVC89BOyYx7aw6qHA2mvyNUjn0g9KiyEz+kC1EF
	rBPrr+D5BqexbkLlFJAMINm+TUtUp4SRkRRzZp7sRN4bgcTJfTg==
X-Google-Smtp-Source: AGHT+IF7kNGSe/wlGT0BWvyBqvSSo7lritKGsYQzZBDuLBeVy5+1nLMmDJ4l0lUwP7hwJrvvIg1mrp/Ra2CJDOHqBQI=
X-Received: by 2002:a2e:9852:0:b0:2d4:6c7d:5750 with SMTP id
 e18-20020a2e9852000000b002d46c7d5750mr519974ljj.39.1710317649155; Wed, 13 Mar
 2024 01:14:09 -0700 (PDT)
MIME-Version: 1.0
References: <bf6b6df0-aa58-44d0-9c59-9a5b022b5620@invisiblethingslab.com> <20240307101901.1137-1-omeg@invisiblethingslab.com>
In-Reply-To: <20240307101901.1137-1-omeg@invisiblethingslab.com>
From: Owen Smith <owen.smith@citrix.com>
Date: Wed, 13 Mar 2024 08:13:57 +0000
Message-ID: <CAC_UY8-SaAFfGQ2N3h7zZ7X5EqL44-k=aAtrfF6ROnx1bF56iA@mail.gmail.com>
Subject: Re: [PATCH] Fix double-free on error condition in GnttabPermitForeignAccess()
To: =?UTF-8?B?UmFmYcWCIFdvamR5xYJh?= <omeg@invisiblethingslab.com>
Cc: win-pv-devel@lists.xenproject.org
Content-Type: multipart/alternative; boundary="000000000000ee81a60613865bcc"

--000000000000ee81a60613865bcc
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Reviewed-by: Owen Smith <owen.smith@cloud.com>

On Thu, Mar 7, 2024 at 10:19=E2=80=AFAM Rafa=C5=82 Wojdy=C5=82a <omeg@invis=
iblethingslab.com>
wrote:

> XENIFACE_GNTTAB_CONTEXT associated with the request was incorrectly freed
> by GnttabPermitForeignAccess() when a failure occured. The context is als=
o
> freed by the parent function, IoctlGnttabPermitForeignAccess(), which led
> to a double-free and kernel heap corruption.
>
> GnttabStopSharing() as the final step in the failure path takes care
> of cleaning up the contents of the (possibly partially initialized)
> context.
>
> Signed-off-by: Rafa=C5=82 Wojdy=C5=82a <omeg@invisiblethingslab.com>
> ---
>  src/xeniface/ioctl_gnttab.c | 6 ------
>  1 file changed, 6 deletions(-)
>
> diff --git a/src/xeniface/ioctl_gnttab.c b/src/xeniface/ioctl_gnttab.c
> index 8ab2099..026f29b 100644
> --- a/src/xeniface/ioctl_gnttab.c
> +++ b/src/xeniface/ioctl_gnttab.c
> @@ -304,12 +304,6 @@ fail2:
>  fail1:
>      Error("Fail1\n");
>      GnttabStopSharing(Fdo, Context, Page);
> -
> -    if (Context !=3D NULL) {
> -        RtlZeroMemory(Context, sizeof(*Context));
> -        __FreePoolWithTag(Context, XENIFACE_POOL_TAG);
> -    }
> -
>      return Status;
>  }
>
> --
> 2.43.0.windows.1
>
>
>

--000000000000ee81a60613865bcc
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_default" style=3D"font-family:monospac=
e">Reviewed-by: Owen Smith &lt;<a href=3D"mailto:owen.smith@cloud.com">owen=
.smith@cloud.com</a>&gt;</div></div><br><div class=3D"gmail_quote"><div dir=
=3D"ltr" class=3D"gmail_attr">On Thu, Mar 7, 2024 at 10:19=E2=80=AFAM Rafa=
=C5=82 Wojdy=C5=82a &lt;<a href=3D"mailto:omeg@invisiblethingslab.com">omeg=
@invisiblethingslab.com</a>&gt; wrote:<br></div><blockquote class=3D"gmail_=
quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,=
204);padding-left:1ex">XENIFACE_GNTTAB_CONTEXT associated with the request =
was incorrectly freed<br>
by GnttabPermitForeignAccess() when a failure occured. The context is also<=
br>
freed by the parent function, IoctlGnttabPermitForeignAccess(), which led<b=
r>
to a double-free and kernel heap corruption.<br>
<br>
GnttabStopSharing() as the final step in the failure path takes care<br>
of cleaning up the contents of the (possibly partially initialized) context=
.<br>
<br>
Signed-off-by: Rafa=C5=82 Wojdy=C5=82a &lt;<a href=3D"mailto:omeg@invisible=
thingslab.com" target=3D"_blank">omeg@invisiblethingslab.com</a>&gt;<br>
---<br>
=C2=A0src/xeniface/ioctl_gnttab.c | 6 ------<br>
=C2=A01 file changed, 6 deletions(-)<br>
<br>
diff --git a/src/xeniface/ioctl_gnttab.c b/src/xeniface/ioctl_gnttab.c<br>
index 8ab2099..026f29b 100644<br>
--- a/src/xeniface/ioctl_gnttab.c<br>
+++ b/src/xeniface/ioctl_gnttab.c<br>
@@ -304,12 +304,6 @@ fail2:<br>
=C2=A0fail1:<br>
=C2=A0 =C2=A0 =C2=A0Error(&quot;Fail1\n&quot;);<br>
=C2=A0 =C2=A0 =C2=A0GnttabStopSharing(Fdo, Context, Page);<br>
-<br>
-=C2=A0 =C2=A0 if (Context !=3D NULL) {<br>
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 RtlZeroMemory(Context, sizeof(*Context));<br>
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 __FreePoolWithTag(Context, XENIFACE_POOL_TAG);=
<br>
-=C2=A0 =C2=A0 }<br>
-<br>
=C2=A0 =C2=A0 =C2=A0return Status;<br>
=C2=A0}<br>
<br>
-- <br>
2.43.0.windows.1<br>
<br>
<br>
</blockquote></div>

--000000000000ee81a60613865bcc--


From win-pv-devel-bounces@lists.xenproject.org Thu Mar 14 08:59:49 2024
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Thu, 14 Mar 2024 08:59:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.693044.1080781 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rkgwF-0005b6-L3; Thu, 14 Mar 2024 08:59:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 693044.1080781; Thu, 14 Mar 2024 08:59:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rkgwF-0005ay-I9; Thu, 14 Mar 2024 08:59:47 +0000
Received: by outflank-mailman (input) for mailman id 693044;
 Thu, 14 Mar 2024 08:59:46 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xOC7=KU=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1rkgwE-0005ao-8V
 for win-pv-devel@lists.xenproject.org; Thu, 14 Mar 2024 08:59:46 +0000
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
 [2a00:1450:4864:20::131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3354726f-e1e1-11ee-afdd-a90da7624cb6;
 Thu, 14 Mar 2024 09:59:45 +0100 (CET)
Received: by mail-lf1-x131.google.com with SMTP id
 2adb3069b0e04-51321e71673so888404e87.2
 for <win-pv-devel@lists.xenproject.org>; Thu, 14 Mar 2024 01:59:44 -0700 (PDT)
Received: from [10.95.157.67] (54-240-197-239.amazon.com. [54.240.197.239])
 by smtp.gmail.com with ESMTPSA id
 m8-20020a05600c4f4800b00413ea26f942sm4478805wmq.14.2024.03.14.01.59.42
 for <win-pv-devel@lists.xenproject.org>
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 14 Mar 2024 01:59:43 -0700 (PDT)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3354726f-e1e1-11ee-afdd-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1710406783; x=1711011583; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:organization:reply-to
         :content-language:references:to:subject:user-agent:mime-version:date
         :message-id:from:from:to:cc:subject:date:message-id:reply-to;
        bh=EUUNknwb1utv8iZdAXapXcf1VVHU428KaZV4oIGPLAI=;
        b=ATLUSWxCOAWhQe0lzRKaJOGLcot5rPfpAWhsn1JLfX+YL/qocYm/JDbQudF4Y7otSw
         tTRcaoG8OIgSqom9clzOevjun1DY4dxAOgNi2GFoOvCvww2hNHAduHGmRoyZ6NsUQ0Fx
         lt8HxKnkiLRGzeTIJGOrhTisqDoBWgi26mNT/dcU/gv5VmrevhR6ETQgu9IhBxfotRMA
         WGmBmcPQseX+ZyuOkd5meHLtU5mOee/eXjTr51JvEf7I2F51vaURYS+HYfTf7GHHStt4
         +MJLDJCJK1ahJrkxa0w4oq3wpRG+xQlnnkuce5cXl3cRB62enC1LIsyy9YdO3ZwmaUw5
         2n0A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1710406783; x=1711011583;
        h=content-transfer-encoding:in-reply-to:organization:reply-to
         :content-language:references:to:subject:user-agent:mime-version:date
         :message-id:from:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=EUUNknwb1utv8iZdAXapXcf1VVHU428KaZV4oIGPLAI=;
        b=EF8wnx5HTyxWpm9KR5ZAHwZLF2GoBzhn8MFWBySTyHwd2yj+3cWTzAYhGRGmrCYbw7
         PUF0eNf+VvVNS2lYw1iHIxR1LhYv7N16Ms7/p1/BVdbRFlavPSxP83ai7l8WvwPxTIVk
         WUxRAomS4mWVu+WjmG4/1MthYG2l/tdCol/HLqO5kez0VOUNyGH3flFedvjYERYzwYgK
         8uKRjsZnzhC9hzayHY2CO1FS+BMByhCP7xo+gYIbAz9rxqlGMGkYJQdKO5PHrQbmz9h4
         G3EyOzR/uGH5+YiBo2ciPKUo+o0Ld5obSgy+jc5L8kxLTElRwg8TnSsAOzRJU1azMeBI
         WH9Q==
X-Gm-Message-State: AOJu0YzEI4PLgVCfiPsCUkdZaGBtC56y6rraRRhhBPygAQnfW5JQDeIn
	o5viURTvqU4rvdgOckzuNEUYSM2CjuQ4uKijc1lRHMQC/OxygYu4a+V6lSpTq3M=
X-Google-Smtp-Source: AGHT+IHdPW3tINVoSlfy/yDBXxe0xSWi1swmQzkP2VPr/bMfH5F4+y2iMoN6e5X1qUK2z8edz/xgKA==
X-Received: by 2002:a05:6512:3687:b0:513:c9f5:cbf2 with SMTP id d7-20020a056512368700b00513c9f5cbf2mr652113lfs.59.1710406783366;
        Thu, 14 Mar 2024 01:59:43 -0700 (PDT)
From: Paul Durrant <xadimgnik@gmail.com>
X-Google-Original-From: Paul Durrant <paul@xen.org>
Message-ID: <43ddc5c7-cacc-4d9e-b879-311c344d85f1@xen.org>
Date: Thu, 14 Mar 2024 08:59:36 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Fix double-free on error condition in
 GnttabPermitForeignAccess()
To: win-pv-devel@lists.xenproject.org
References: <bf6b6df0-aa58-44d0-9c59-9a5b022b5620@invisiblethingslab.com>
 <20240307101901.1137-1-omeg@invisiblethingslab.com>
 <CAC_UY8-SaAFfGQ2N3h7zZ7X5EqL44-k=aAtrfF6ROnx1bF56iA@mail.gmail.com>
Content-Language: en-US
Reply-To: paul@xen.org
Organization: Xen Project
In-Reply-To: <CAC_UY8-SaAFfGQ2N3h7zZ7X5EqL44-k=aAtrfF6ROnx1bF56iA@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 13/03/2024 08:13, Owen Smith wrote:
> Reviewed-by: Owen Smith <owen.smith@cloud.com <mailto:owen.smith@cloud.com>>
> 

Acked-by: Paul Durrant <paul@xen.org>

> On Thu, Mar 7, 2024 at 10:19 AM Rafał Wojdyła 
> <omeg@invisiblethingslab.com <mailto:omeg@invisiblethingslab.com>> wrote:
> 
>     XENIFACE_GNTTAB_CONTEXT associated with the request was incorrectly
>     freed
>     by GnttabPermitForeignAccess() when a failure occured. The context
>     is also
>     freed by the parent function, IoctlGnttabPermitForeignAccess(),
>     which led
>     to a double-free and kernel heap corruption.
> 
>     GnttabStopSharing() as the final step in the failure path takes care
>     of cleaning up the contents of the (possibly partially initialized)
>     context.
> 
>     Signed-off-by: Rafał Wojdyła <omeg@invisiblethingslab.com
>     <mailto:omeg@invisiblethingslab.com>>

Not sure why this ended up with the 'mailto' part; I'll delete it on commit.

>     ---
>       src/xeniface/ioctl_gnttab.c | 6 ------
>       1 file changed, 6 deletions(-)
> 
>     diff --git a/src/xeniface/ioctl_gnttab.c b/src/xeniface/ioctl_gnttab.c
>     index 8ab2099..026f29b 100644
>     --- a/src/xeniface/ioctl_gnttab.c
>     +++ b/src/xeniface/ioctl_gnttab.c
>     @@ -304,12 +304,6 @@ fail2:
>       fail1:
>           Error("Fail1\n");
>           GnttabStopSharing(Fdo, Context, Page);
>     -
>     -    if (Context != NULL) {
>     -        RtlZeroMemory(Context, sizeof(*Context));
>     -        __FreePoolWithTag(Context, XENIFACE_POOL_TAG);
>     -    }
>     -
>           return Status;
>       }
> 
>     -- 
>     2.43.0.windows.1
> 
> 



From win-pv-devel-bounces@lists.xenproject.org Thu Mar 14 09:45:20 2024
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Thu, 14 Mar 2024 09:45:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.693075.1080818 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rkheJ-0005HC-H8; Thu, 14 Mar 2024 09:45:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 693075.1080818; Thu, 14 Mar 2024 09:45:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rkheJ-0005H5-EI; Thu, 14 Mar 2024 09:45:19 +0000
Received: by outflank-mailman (input) for mailman id 693075;
 Thu, 14 Mar 2024 09:45:18 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xOC7=KU=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1rkheI-0005Gz-65
 for win-pv-devel@lists.xenproject.org; Thu, 14 Mar 2024 09:45:18 +0000
Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com
 [2a00:1450:4864:20::329])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 906a8dba-e1e7-11ee-afdd-a90da7624cb6;
 Thu, 14 Mar 2024 10:45:17 +0100 (CET)
Received: by mail-wm1-x329.google.com with SMTP id
 5b1f17b1804b1-413f2f3e5b5so1823685e9.3
 for <win-pv-devel@lists.xenproject.org>; Thu, 14 Mar 2024 02:45:17 -0700 (PDT)
Received: from [10.95.157.67] (54-240-197-239.amazon.com. [54.240.197.239])
 by smtp.gmail.com with ESMTPSA id
 z22-20020a05600c0a1600b00413f04cb3b2sm2130979wmp.0.2024.03.14.02.45.15
 for <win-pv-devel@lists.xenproject.org>
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 14 Mar 2024 02:45:15 -0700 (PDT)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 906a8dba-e1e7-11ee-afdd-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1710409517; x=1711014317; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:organization:reply-to
         :references:to:content-language:subject:user-agent:mime-version:date
         :message-id:from:from:to:cc:subject:date:message-id:reply-to;
        bh=Tq7GL3yWKH/SPb06Rz3kf1eTVtFiE/ASmUuceUGGGMo=;
        b=hFj3TNILAG68rks6Dvuw//MVJquvDeVendAF0u1Xt1oj9c2QVyBxYiWEi3sYfM0HVg
         gVQZfHaVKDjG/4E0nkpcnORoaLISeltLzCLXY6/7PGjmyLMYKNFRfhT+E00JjcmN7eyK
         8o2P44cfsakK3DlpLqHxtpZwImNcfVRXStmwW+TV8i5S0l/RXozkq3GXsun5iIqPCLqe
         X6fNti/V59tJNioOsq2QpLv2Q3zs2h9IHtComjYyJygkdGON3jTKFxeLQXea08kvUtbr
         JQxobK9HyEj57UHHx4L2tYGMIPblSG0vf5qECZlBuNX6D2Ishh6LsQnn3G3a3sfr8IWx
         M+3Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1710409517; x=1711014317;
        h=content-transfer-encoding:in-reply-to:organization:reply-to
         :references:to:content-language:subject:user-agent:mime-version:date
         :message-id:from:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Tq7GL3yWKH/SPb06Rz3kf1eTVtFiE/ASmUuceUGGGMo=;
        b=wzM5MvWTTjlkSpWy9Fx+pt8xdcNazS2giz2CmS9FYQ1C9Y1yVUZzc9f3dNTRKfWZxe
         j9aAjpDKzM1P2UVRvzCXPlG3X1JeOLC05HSi0C9AJYO+YwFEyadJmk/LvWc6QwpLVk17
         Y9So4p9zxqg58Ah9eQBn5clu2/D7BOf0DaSP9FklGXNuVzUqU0MwaDQSbNBGHxT+R7O+
         GoeoiNSFGoT/yrtLz1oE4h/Fxe5uK2cy1ougDKGD5+c+qdPb7UwF4C6QFO0/ZnPc2Mll
         EFcRarMajBpI7s2kx+M5cY8TxK2LZMfm1UYpMqHPRKrl81qa8+ZuEsQqGxisOqPuSdUP
         iTAw==
X-Gm-Message-State: AOJu0Yx0Clgm2Apau8ld9+DAwUpIJ3Bqw38JpXbh3qM1P19J/pfNzpmh
	8sQHf46nIlW9UcwXswqhQl9MFDZ1qzD6rX+3A3nKnYoxXCOJY5cIDO8SaHyUB80=
X-Google-Smtp-Source: AGHT+IFSEcJjb9AKtr3tRKMcFBrtMtvirtVa7xP5f9bPetu47v2zKi91HzpAkfM3aTZPQjzVwFeQ1w==
X-Received: by 2002:a05:600c:458b:b0:413:e523:eed2 with SMTP id r11-20020a05600c458b00b00413e523eed2mr1076642wmo.33.1710409516183;
        Thu, 14 Mar 2024 02:45:16 -0700 (PDT)
From: Paul Durrant <xadimgnik@gmail.com>
X-Google-Original-From: Paul Durrant <paul@xen.org>
Message-ID: <688613fd-1710-4ece-b315-f6960a3dc8ec@xen.org>
Date: Thu, 14 Mar 2024 09:45:09 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Asynchronous power handling.
Content-Language: en-US
To: win-pv-devel@lists.xenproject.org
References: <20240312072352.1363-1-owen.smith@cloud.com>
Reply-To: paul@xen.org
Organization: Xen Project
In-Reply-To: <20240312072352.1363-1-owen.smith@cloud.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 12/03/2024 07:23, Owen Smith wrote:
> From: Martin Harvey <Martin.Harvey@citrix.com>
> 
> XenDisk requires minimal IRP_MN_SET_POWER/IRP_MN_QUERY_POWER interactions.
> No IoWorkItems are required as operations perform no significant work.
> Power handlers are is limited to tracking state changes and calling PoSetPowerState.
> 
> Signed-off-by: Martin Harvey <martin.harvey@citrix.com>
> 
> Refactored
> Signed-off-by: Owen Smith <owen.smith@cloud.com>
> 
> ---
>   src/xendisk/fdo.c | 552 +++++++++------------------------------------
>   src/xendisk/pdo.c | 555 +++++++++-------------------------------------
>   2 files changed, 209 insertions(+), 898 deletions(-)
> 
> diff --git a/src/xendisk/fdo.c b/src/xendisk/fdo.c
> index 056e24a..2e5c0de 100644
> --- a/src/xendisk/fdo.c
> +++ b/src/xendisk/fdo.c
[snip]
> @@ -1216,13 +1219,12 @@ __FdoSetDevicePower(
>             PowerActionName(PowerAction));
>   
>       if (DeviceState == __FdoGetDevicePowerState(Fdo)) {
> -        status = FdoForwardIrpSynchronously(Fdo, Irp);
> -        IoCompleteRequest(Irp, IO_NO_INCREMENT);
> -
> +        IoSkipCurrentIrpStackLocation(Irp);
> +        status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
>           goto done;
>       }
>   
> -    status = (DeviceState < __FdoGetDevicePowerState(Fdo)) ?

This appears to be an unnecessary change.

> +    status = DeviceState < __FdoGetDevicePowerState(Fdo) ?
>                __FdoSetDevicePowerUp(Fdo, Irp) :
>                __FdoSetDevicePowerDown(Fdo, Irp);
>   
[snip]
> @@ -1314,13 +1318,12 @@ __FdoSetSystemPower(
>             PowerActionName(PowerAction));
>   
>       if (SystemState == __FdoGetSystemPowerState(Fdo)) {
> -        status = FdoForwardIrpSynchronously(Fdo, Irp);
> -        IoCompleteRequest(Irp, IO_NO_INCREMENT);
> -
> +        IoSkipCurrentIrpStackLocation(Irp);
> +        status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
>           goto done;
>       }
>   
> -    status = (SystemState < __FdoGetSystemPowerState(Fdo)) ?

Same here.

I can drop them on commit.

Reviewed-by: Paul Durrant <paul@xen.org>

> +    status = SystemState < __FdoGetSystemPowerState(Fdo) ?
>                __FdoSetSystemPowerUp(Fdo, Irp) :
>                __FdoSetSystemPowerDown(Fdo, Irp);
>   



From win-pv-devel-bounces@lists.xenproject.org Thu Mar 14 10:30:55 2024
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Thu, 14 Mar 2024 10:30:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.693111.1080861 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rkiMP-0005nK-IK; Thu, 14 Mar 2024 10:30:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 693111.1080861; Thu, 14 Mar 2024 10:30:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rkiMP-0005nD-Fd; Thu, 14 Mar 2024 10:30:53 +0000
Received: by outflank-mailman (input) for mailman id 693111;
 Thu, 14 Mar 2024 10:30:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xOC7=KU=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1rkiMN-0005n5-L7
 for win-pv-devel@lists.xenproject.org; Thu, 14 Mar 2024 10:30:52 +0000
Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com
 [2a00:1450:4864:20::32e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ecafe3fa-e1ed-11ee-afdd-a90da7624cb6;
 Thu, 14 Mar 2024 11:30:49 +0100 (CET)
Received: by mail-wm1-x32e.google.com with SMTP id
 5b1f17b1804b1-413e93b0f54so5231685e9.3
 for <win-pv-devel@lists.xenproject.org>; Thu, 14 Mar 2024 03:30:49 -0700 (PDT)
Received: from [10.95.157.67] (54-240-197-239.amazon.com. [54.240.197.239])
 by smtp.gmail.com with ESMTPSA id
 o18-20020a05600c4fd200b004132f9cf053sm5155658wmq.33.2024.03.14.03.30.47
 for <win-pv-devel@lists.xenproject.org>
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 14 Mar 2024 03:30:47 -0700 (PDT)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ecafe3fa-e1ed-11ee-afdd-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1710412248; x=1711017048; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:organization:reply-to
         :references:to:content-language:subject:user-agent:mime-version:date
         :message-id:from:from:to:cc:subject:date:message-id:reply-to;
        bh=V4I7Xr+1z6pr3MXEoMDGsnes9Yx8oTjDC8D4MwBJ6RQ=;
        b=RQA1HCFTd6ok3u3xaq1ajxFWRbFzTZIiFd6MQSjcKmwunouzLNIDbnK6CCbsYCYiuy
         Bs8MvjZ4GkktZVaCaK3WE1fnWHXBDMw0HlCFWgMnA+C4gUzlYXT6qpOiVk2Qqi7481gd
         g/tNj6kk67NUHJxQnqkz1Wnd4wae4f5/9vVsNkAuYoDUk7vP4tTJF1xygU3AgwW70vNA
         OB4wHYBAoajh/LEcF8Sns830JeuX3p6RAl1210gYMPa8gTbnaE71wGNMZbBiRv870tp4
         fYFHmu6He6kLGEVLoURn3kNi2jDBG/6e7laPFXDgFkXrzB5L0QvhFg/+Iw4L2MGfJqgO
         ah6w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1710412248; x=1711017048;
        h=content-transfer-encoding:in-reply-to:organization:reply-to
         :references:to:content-language:subject:user-agent:mime-version:date
         :message-id:from:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=V4I7Xr+1z6pr3MXEoMDGsnes9Yx8oTjDC8D4MwBJ6RQ=;
        b=W52s2NA20IqLKfJtdCNJB6LqNAk5NffTCjIn1i3YJX9VSAHg0KdSsrNgeYXG5wtSi7
         bn0JDRlflhmANyp1oQ8DdiQkzdyMBVwaIMIQKKfe9e/gg/OmtYWS69zirWMnjFuMpmoT
         BnQ8Ydaec2kHarOQz7sibGaUshZM3GR0TvBzVxfs9fZxC++r0do5i+mVtukRTvDrAnQ0
         vu2VIuHRDkEZCgBFb8UXzHbSyFQ36FH2S6zGF1Hws59XJOZggo2Fwle18LO6mvZ+G3aD
         lTQdPbF6MbS4nSfAN5caJuWWAmx3zhffumTqRu2OjmaNmgxMiFD/029I4SRPKDvJ9du6
         r4uQ==
X-Gm-Message-State: AOJu0YxQq1W61n8ngY65r7wDpa+8MxnB0kjBWERu3rkn2Qn7w8XJe6yE
	gCjHX9OE6h4VmpUL/J4JKovKF3vt199D1Pl3m6B9WsHlhJr3l2txBKALhinr4ik=
X-Google-Smtp-Source: AGHT+IFhqSUtgMZ7V9IJNtSZocF/4riNVwpGvx7+5KBvNAW7BOWs+x3eEybHYohLGYvcy3OcwBSPtQ==
X-Received: by 2002:a05:600c:310d:b0:413:eae0:1ed1 with SMTP id g13-20020a05600c310d00b00413eae01ed1mr1060179wmo.25.1710412247837;
        Thu, 14 Mar 2024 03:30:47 -0700 (PDT)
From: Paul Durrant <xadimgnik@gmail.com>
X-Google-Original-From: Paul Durrant <paul@xen.org>
Message-ID: <d60c8263-1928-4689-b64f-be22a5e2f9a1@xen.org>
Date: Thu, 14 Mar 2024 10:30:41 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Asynchronous power handling.
Content-Language: en-US
To: win-pv-devel@lists.xenproject.org
References: <20240312072333.787-1-owen.smith@cloud.com>
Reply-To: paul@xen.org
Organization: Xen Project
In-Reply-To: <20240312072333.787-1-owen.smith@cloud.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 12/03/2024 07:23, Owen Smith wrote:
> From: Martin Harvey <Martin.Harvey@citrix.com>
> 
> Use completion routines and IO_WORKITEMs to handle power state transitions,
> allowing execution to run at PASSIVE_LEVEL without blocking the current thread,
> and avoid blocking all power IRPs while another is executing (power IRPs should
> be synchronized by the power manager, but this doesnt appear to be the case in
> all situations).
> 
> Signed-off-by: Martin Harvey <martin.harvey@citrix.com>
> 
> Refactored

Unfortunately, not enough...

> Signed-off-by: Owen Smith <owen.smith@cloud.com>
> 
> ---
>   src/xenvif/fdo.c | 803 +++++++++++++++++++++++++++++------------------
>   src/xenvif/pdo.c | 263 +++++++---------
>   2 files changed, 608 insertions(+), 458 deletions(-)
> 
> diff --git a/src/xenvif/fdo.c b/src/xenvif/fdo.c
> index 451d582..7fa2d04 100644
> --- a/src/xenvif/fdo.c
> +++ b/src/xenvif/fdo.c
> @@ -81,9 +81,9 @@ struct _XENVIF_FDO {
>       ULONG                       Usage[DeviceUsageTypeDumpFile + 1];
>       BOOLEAN                     NotDisableable;
>   
> -    PXENVIF_THREAD              SystemPowerThread;
> +    PIO_WORKITEM                SystemPowerWorkItem;
>       PIRP                        SystemPowerIrp;
> -    PXENVIF_THREAD              DevicePowerThread;
> +    PIO_WORKITEM                DevicePowerWorkItem;
>       PIRP                        DevicePowerIrp;
>   
>       CHAR                        VendorName[MAXNAMELEN];
> @@ -2168,6 +2168,68 @@ FdoDispatchPnp(
>       return status;
>   }
>   
> +__drv_functionClass(IO_WORKITEM_ROUTINE)
> +__drv_sameIRQL
> +static VOID
> +FdoDevicePowerUpWorker(
> +    IN  PDEVICE_OBJECT  DeviceObject,
> +    IN  PVOID           Context
> +    )
> +{
> +    PXENVIF_FDO         Fdo = (PXENVIF_FDO) Context;
> +    PIRP                Irp;
> +
> +    UNREFERENCED_PARAMETER(DeviceObject);
> +
> +    Irp = InterlockedExchangePointer(&Fdo->DevicePowerIrp, NULL);
> +    ASSERT(Irp != NULL);
> +
> +    (VOID) FdoD3ToD0(Fdo);
> +
> +    /* Cannot change Irp->IoStatus */
> +    /* Continue completion chain */
> +    IoCompleteRequest(Irp, IO_NO_INCREMENT);
> +}
> +
> +__drv_functionClass(IO_COMPLETION_ROUTINE)
> +__drv_sameIRQL
> +static NTSTATUS
> +FdoSetDevicePowerUpComplete(
> +    IN  PDEVICE_OBJECT  DeviceObject,
> +    IN  PIRP            Irp,
> +    IN  PVOID           Context
> +    )
> +{
> +    PXENVIF_FDO         Fdo = (PXENVIF_FDO) Context;
> +    PIO_STACK_LOCATION  StackLocation;
> +    DEVICE_POWER_STATE  DeviceState;
> +    PVOID               Exchange;
> +
> +    UNREFERENCED_PARAMETER(DeviceObject);
> +
> +    StackLocation = IoGetCurrentIrpStackLocation(Irp);
> +    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
> +
> +    /* Already marked pending by us */
> +    Info("%s -> %s\n",

Lack of device name in the Info() message make it less useful

> +         PowerDeviceStateName(__FdoGetDevicePowerState(Fdo)),
> +         PowerDeviceStateName(DeviceState));
> +
> +    /* Don't worry about IRP IoStatus */
> +    ASSERT3U(DeviceState, ==, PowerDeviceD0);
> +
> +    Exchange = InterlockedExchangePointer(&Fdo->DevicePowerIrp, Irp);
> +    ASSERT(Exchange == NULL);
> +
> +    IoQueueWorkItem(Fdo->DevicePowerWorkItem,
> +                    FdoDevicePowerUpWorker,
> +                    DelayedWorkQueue,
> +                    Fdo);
> +
> +    /* Stop completion chain in all circumstances. */
> +    return STATUS_MORE_PROCESSING_REQUIRED;
> +}
> +
>   static FORCEINLINE NTSTATUS
>   __FdoSetDevicePowerUp(
>       IN  PXENVIF_FDO     Fdo,
> @@ -2176,7 +2238,6 @@ __FdoSetDevicePowerUp(
>   {
>       PIO_STACK_LOCATION  StackLocation;
>       DEVICE_POWER_STATE  DeviceState;
> -    NTSTATUS            status;
>   
>       Trace("====>\n");
>   
> @@ -2185,25 +2246,41 @@ __FdoSetDevicePowerUp(
>   
>       ASSERT3U(DeviceState, <,  __FdoGetDevicePowerState(Fdo));
>   
> -    status = FdoForwardIrpSynchronously(Fdo, Irp);
> -    if (!NT_SUCCESS(status))
> -        goto done;
> +    IoMarkIrpPending(Irp);
> +    IoCopyCurrentIrpStackLocationToNext(Irp);
> +    IoSetCompletionRoutine(Irp,
> +                           FdoSetDevicePowerUpComplete,
> +                           Fdo,
> +                           TRUE,
> +                           TRUE,
> +                           TRUE);
>   
> -    Info("%s: %s -> %s\n",
> -         __FdoGetName(Fdo),
> -         PowerDeviceStateName(__FdoGetDevicePowerState(Fdo)),
> -         PowerDeviceStateName(DeviceState));
> +    (VOID) IoCallDriver(Fdo->LowerDeviceObject, Irp);
>   
> -    ASSERT3U(DeviceState, ==, PowerDeviceD0);
> -    status = FdoD3ToD0(Fdo);
> -    ASSERT(NT_SUCCESS(status));
> +    return STATUS_PENDING;
> +}
>   
> -done:
> -    Irp->IoStatus.Status = status;
> -    IoCompleteRequest(Irp, IO_NO_INCREMENT);
> +__drv_functionClass(IO_WORKITEM_ROUTINE)
> +__drv_sameIRQL
> +static VOID
> +FdoDevicePowerDownWorker(
> +    IN  PDEVICE_OBJECT  DeviceObject,
> +    IN  PVOID           Context
> +    )
> +{
> +    PXENVIF_FDO         Fdo = (PXENVIF_FDO) Context;
> +    PIRP                Irp;
>   
> -    Trace("<==== (%08x)\n", status);
> -    return status;
> +    UNREFERENCED_PARAMETER(DeviceObject);
> +
> +    Irp = InterlockedExchangePointer(&Fdo->DevicePowerIrp, NULL);
> +    ASSERT(Irp != NULL);
> +
> +    FdoD0ToD3(Fdo);
> +
> +    /* We are on dispatch path here. */
> +    IoCopyCurrentIrpStackLocationToNext(Irp);
> +    IoCallDriver(Fdo->LowerDeviceObject, Irp);
>   }
>   
>   static FORCEINLINE NTSTATUS
> @@ -2221,18 +2298,32 @@ __FdoSetDevicePowerDown(
>   
>       ASSERT3U(DeviceState, >,  __FdoGetDevicePowerState(Fdo));
>   
> -    Info("%s: %s -> %s\n",
> -         __FdoGetName(Fdo),
> +    Info("%s -> %s\n",

Device name being gratuitously removed here.

>            PowerDeviceStateName(__FdoGetDevicePowerState(Fdo)),
>            PowerDeviceStateName(DeviceState));
>   
>       ASSERT3U(DeviceState, ==, PowerDeviceD3);
>   
> -    if (__FdoGetDevicePowerState(Fdo) == PowerDeviceD0)
> -        FdoD0ToD3(Fdo);
> +    /* All handled in dispatch, no extra error handling */
> +    if (__FdoGetDevicePowerState(Fdo) != PowerDeviceD0) {
> +        /* Not marked pending, can skip */

Inconsistent comment format.

> +        IoSkipCurrentIrpStackLocation(Irp);
> +        status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
> +    } else {
> +        PVOID           Exchange;
>   
> -    IoSkipCurrentIrpStackLocation(Irp);
> -    status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
> +        IoMarkIrpPending(Irp);
> +
> +        Exchange = InterlockedExchangePointer(&Fdo->DevicePowerIrp, Irp);
> +        ASSERT(Exchange == NULL);
> +
> +        IoQueueWorkItem(Fdo->DevicePowerWorkItem,
> +                        FdoDevicePowerDownWorker,
> +                        DelayedWorkQueue,
> +                        Fdo);
> +
> +        status = STATUS_PENDING;
> +    }
>   
>       return status;
>   }
> @@ -2253,11 +2344,9 @@ __FdoSetDevicePower(
>       PowerAction = StackLocation->Parameters.Power.ShutdownType;
>   
>       Trace("====> (%s:%s)\n",
> -          PowerDeviceStateName(DeviceState),
> +          PowerDeviceStateName(DeviceState),
>             PowerActionName(PowerAction));
>   
> -    ASSERT3U(PowerAction, <, PowerActionShutdown);
> -
>       if (DeviceState == __FdoGetDevicePowerState(Fdo)) {
>           IoSkipCurrentIrpStackLocation(Irp);
>           status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
> @@ -2271,7 +2360,7 @@ __FdoSetDevicePower(
>   
>   done:
>       Trace("<==== (%s:%s)(%08x)\n",
> -          PowerDeviceStateName(DeviceState),
> +          PowerDeviceStateName(DeviceState),
>             PowerActionName(PowerAction),
>             status);
>       return status;
> @@ -2279,8 +2368,8 @@ done:
>   
>   __drv_functionClass(REQUEST_POWER_COMPLETE)
>   __drv_sameIRQL
> -VOID
> -__FdoRequestSetDevicePower(
> +static VOID
> +FdoRequestSetDevicePowerUpComplete(
>       IN  PDEVICE_OBJECT      DeviceObject,
>       IN  UCHAR               MinorFunction,
>       IN  POWER_STATE         PowerState,
> @@ -2288,47 +2377,130 @@ __FdoRequestSetDevicePower(
>       IN  PIO_STATUS_BLOCK    IoStatus
>       )
>   {
> -    PKEVENT                 Event = Context;
> +    PIRP                    Irp = (PIRP) Context;
>   
>       UNREFERENCED_PARAMETER(DeviceObject);
>       UNREFERENCED_PARAMETER(MinorFunction);
>       UNREFERENCED_PARAMETER(PowerState);
> +    UNREFERENCED_PARAMETER(IoStatus);
>   
> -    ASSERT(NT_SUCCESS(IoStatus->Status));
> -
> -    KeSetEvent(Event, IO_NO_INCREMENT, FALSE);
> +    /* Although can change Irp->IoStatus for pended IRP, drivers should not fail this. */

Again, wrong comment style.

> +    IoCompleteRequest(Irp, IO_NO_INCREMENT);
>   }
>   
> +__drv_functionClass(IO_WORKITEM_ROUTINE)
> +__drv_sameIRQL
>   static VOID
> -FdoRequestSetDevicePower(
> -    IN  PXENVIF_FDO         Fdo,
> -    IN  DEVICE_POWER_STATE  DeviceState
> +FdoSystemPowerUpWorker(
> +    IN  PDEVICE_OBJECT  DeviceObject,
> +    IN  PVOID           Context
>       )
>   {
> -    POWER_STATE             PowerState;
> -    KEVENT                  Event;
> -    NTSTATUS                status;
> +    PXENVIF_FDO         Fdo = (PXENVIF_FDO) Context;
> +    PIRP                Irp;
> +    PIO_STACK_LOCATION  StackLocation;
> +    SYSTEM_POWER_STATE  SystemState;
> +    POWER_STATE         PowerState;
> +    NTSTATUS            status;
>   
> -    Trace("%s\n", PowerDeviceStateName(DeviceState));
> +    UNREFERENCED_PARAMETER(DeviceObject);
>   
> -    ASSERT3U(KeGetCurrentIrql(), ==, PASSIVE_LEVEL);
> +    Irp = InterlockedExchangePointer(&Fdo->SystemPowerIrp, NULL);
> +    ASSERT(Irp != NULL);
>   
> -    PowerState.DeviceState = DeviceState;
> -    KeInitializeEvent(&Event, NotificationEvent, FALSE);
> +    StackLocation = IoGetCurrentIrpStackLocation(Irp);;
> +    SystemState = StackLocation->Parameters.Power.State.SystemState;
> +
> +    Info("%s -> %s\n",

Lack of device name in Info() again.

> +         PowerSystemStateName(__FdoGetSystemPowerState(Fdo)),
> +         PowerSystemStateName(SystemState));
> +
> +    __FdoSetSystemPowerState(Fdo, PowerSystemHibernate);
> +    FdoS4ToS3(Fdo);
> +    __FdoSetSystemPowerState(Fdo, SystemState);
> +
> +    PowerState.DeviceState = Fdo->LowerDeviceCapabilities.DeviceState[SystemState];
>   
>       status = PoRequestPowerIrp(Fdo->LowerDeviceObject,
>                                  IRP_MN_SET_POWER,
>                                  PowerState,
> -                               __FdoRequestSetDevicePower,
> -                               &Event,
> +                               FdoRequestSetDevicePowerUpComplete,
> +                               Irp,
>                                  NULL);
> -    ASSERT(NT_SUCCESS(status));
>   
> -    (VOID) KeWaitForSingleObject(&Event,
> -                                 Executive,
> -                                 KernelMode,
> -                                 FALSE,
> -                                 NULL);
> +    /* Whatever happens, we have stopped completion processing for SIrp,
> +      to restart SIrp processing, call IoCompleteRequest. */

And again. In fact they are everywhere; please remove them or fix them. 
TBH it looks like many of them are OTT and can be removed.

> +    if (!NT_SUCCESS(status))
> +        goto fail1;
> +
> +    return; /* FdoRequestSetDevicePowerUpComplete will complete the request.*/
> +
> +fail1:
> +    Error("fail1 - but continue IRP processing. (%08x)\n", status);
> +    /* Although can change Irp->IoStatus for pended IRP, drivers should not fail this. */
> +    IoCompleteRequest(Irp, IO_NO_INCREMENT);
> +}
> +
> +__drv_functionClass(IO_COMPLETION_ROUTINE)
> +__drv_sameIRQL
> +static NTSTATUS
> +FdoSetSystemPowerUpComplete(
> +    IN  PDEVICE_OBJECT  DeviceObject,
> +    IN  PIRP            Irp,
> +    IN  PVOID           Context
> +    )
> +{
> +    PXENVIF_FDO         Fdo = (PXENVIF_FDO) Context;
> +    PIO_STACK_LOCATION  StackLocation;
> +    SYSTEM_POWER_STATE  SystemState;
> +    POWER_STATE         PowerState;
> +    NTSTATUS            status;
> +
> +    UNREFERENCED_PARAMETER(DeviceObject);
> +
> +    /* IRP marked as pending on dispatch path, no need to check pending returned */
> +
> +    /* Don't worry about IRP IoStatus */
> +    StackLocation = IoGetCurrentIrpStackLocation(Irp);
> +    SystemState = StackLocation->Parameters.Power.State.SystemState;
> +
> +    if (SystemState < PowerSystemHibernate &&
> +        __FdoGetSystemPowerState(Fdo) >= PowerSystemHibernate) {
> +        PVOID           Exchange;
> +
> +        Exchange = InterlockedExchangePointer(&Fdo->SystemPowerIrp, Irp);
> +        ASSERT(Exchange == NULL);
> +
> +        IoQueueWorkItem(Fdo->SystemPowerWorkItem,
> +                        FdoSystemPowerUpWorker,
> +                        DelayedWorkQueue,
> +                        Fdo);
> +    } else {
> +        Info("%s -> %s\n",

Lack of device name in Info() again.

> +             PowerSystemStateName(__FdoGetSystemPowerState(Fdo)),
> +             PowerSystemStateName(SystemState));
> +
> +        __FdoSetSystemPowerState(Fdo, SystemState);
> +
> +        PowerState.DeviceState = Fdo->LowerDeviceCapabilities.DeviceState[SystemState];
> +
> +        status = PoRequestPowerIrp(Fdo->LowerDeviceObject,
> +                                   IRP_MN_SET_POWER,
> +                                   PowerState,
> +                                   FdoRequestSetDevicePowerUpComplete,
> +                                   Irp,
> +                                   NULL);
> +        if (!NT_SUCCESS(status))
> +            goto fail1;
> +    }
> +
> +    /* Will later complete the IRP (which is marked pending, in worker or 2nd level comp routine.)*/
> +    return STATUS_MORE_PROCESSING_REQUIRED;
> +
> +fail1:
> +    Error("fail1 - but continue IRP processing. (%08x)\n", status);
> +    /* marked pending on dispatch path, don't change Irp->IoStatus, keep going.*/
> +    return STATUS_CONTINUE_COMPLETION;
>   }
>   
>   static FORCEINLINE NTSTATUS
> @@ -2337,42 +2509,103 @@ __FdoSetSystemPowerUp(
>       IN  PIRP            Irp
>       )
>   {
> -
>       PIO_STACK_LOCATION  StackLocation;
>       SYSTEM_POWER_STATE  SystemState;
> -    DEVICE_POWER_STATE  DeviceState;
> -    NTSTATUS            status;
>   
>       StackLocation = IoGetCurrentIrpStackLocation(Irp);
>       SystemState = StackLocation->Parameters.Power.State.SystemState;
>   
>       ASSERT3U(SystemState, <,  __FdoGetSystemPowerState(Fdo));
>   
> -    status = FdoForwardIrpSynchronously(Fdo, Irp);
> -    if (!NT_SUCCESS(status))
> -        goto done;
> +    /* IRP already pended, and we want to complete it *after* completion routine. */
> +    IoCopyCurrentIrpStackLocationToNext(Irp);
> +    IoSetCompletionRoutine(Irp,
> +                           FdoSetSystemPowerUpComplete,
> +                           Fdo,
> +                           TRUE,
> +                           TRUE,
> +                           TRUE);
>   
> -    Info("%s: %s -> %s\n",
> -         __FdoGetName(Fdo),
> -         PowerSystemStateName(__FdoGetSystemPowerState(Fdo)),
> -         PowerSystemStateName(SystemState));
> +    IoCallDriver(Fdo->LowerDeviceObject, Irp);
>   
> -    if (SystemState < PowerSystemHibernate &&
> -        __FdoGetSystemPowerState(Fdo) >= PowerSystemHibernate) {
> -        __FdoSetSystemPowerState(Fdo, PowerSystemHibernate);
> -        FdoS4ToS3(Fdo);
> -    }
> +    return STATUS_PENDING;
> +}
> +
> +__drv_functionClass(IO_WORKITEM_ROUTINE)
> +__drv_sameIRQL
> +static VOID
> +FdoSystemPowerDownWorker(
> +    IN  PDEVICE_OBJECT  DeviceObject,
> +    IN  PVOID           Context
> +    )
> +{
> +    PXENVIF_FDO         Fdo = (PXENVIF_FDO) Context;
> +    PIRP                Irp;
> +    PIO_STACK_LOCATION  StackLocation;
> +    SYSTEM_POWER_STATE  SystemState;
> +
> +    UNREFERENCED_PARAMETER(DeviceObject);
> +
> +    Irp = InterlockedExchangePointer(&Fdo->SystemPowerIrp, NULL);
> +    ASSERT(Irp != NULL);
>   
> +    StackLocation = IoGetCurrentIrpStackLocation(Irp);
> +    SystemState = StackLocation->Parameters.Power.State.SystemState;
> +
> +    __FdoSetSystemPowerState(Fdo, PowerSystemSleeping3);
> +    FdoS3ToS4(Fdo);
>       __FdoSetSystemPowerState(Fdo, SystemState);
>   
> -    DeviceState = Fdo->LowerDeviceCapabilities.DeviceState[SystemState];
> -    FdoRequestSetDevicePower(Fdo, DeviceState);
> +    IoCopyCurrentIrpStackLocationToNext(Irp); /* Irp has been pended */
> +    IoCallDriver(Fdo->LowerDeviceObject, Irp);
> +}
>   
> -done:
> -    Irp->IoStatus.Status = status;
> -    IoCompleteRequest(Irp, IO_NO_INCREMENT);
> +__drv_functionClass(REQUEST_POWER_COMPLETE)
> +__drv_sameIRQL
> +static VOID
> +FdoRequestSetDevicePowerDownComplete(
> +    IN  PDEVICE_OBJECT      DeviceObject,
> +    IN  UCHAR               MinorFunction,
> +    IN  POWER_STATE         PowerState,
> +    IN  PVOID               Context,
> +    IN  PIO_STATUS_BLOCK    IoStatus
> +    )
> +{
> +    PIRP                    Irp = (PIRP) Context;
> +    PIO_STACK_LOCATION      StackLocation = IoGetCurrentIrpStackLocation(Irp);
> +    PDEVICE_OBJECT          UpperDeviceObject = StackLocation->DeviceObject;
> +    PXENVIF_DX              Dx = (PXENVIF_DX)UpperDeviceObject->DeviceExtension;
> +    PXENVIF_FDO             Fdo = Dx->Fdo;
> +    SYSTEM_POWER_STATE      SystemState = StackLocation->Parameters.Power.State.SystemState;
>   
> -    return status;
> +    UNREFERENCED_PARAMETER(DeviceObject);
> +    UNREFERENCED_PARAMETER(MinorFunction);
> +    UNREFERENCED_PARAMETER(PowerState);
> +
> +    if (!NT_SUCCESS(IoStatus->Status))
> +        Error("fail1 - but continue IRP processing. (%08x)\n", IoStatus->Status);
> +
> +    Info("%s -> %s\n",
> +        PowerSystemStateName(__FdoGetSystemPowerState(Fdo)),
> +        PowerSystemStateName(SystemState));
> +
> +    if (SystemState >= PowerSystemHibernate &&
> +        __FdoGetSystemPowerState(Fdo) < PowerSystemHibernate) {
> +        PVOID Exchange;
> +
> +        Exchange = InterlockedExchangePointer(&Fdo->SystemPowerIrp, Irp);
> +        ASSERT(Exchange == NULL);
> +
> +        IoQueueWorkItem(Fdo->DevicePowerWorkItem,
> +                        FdoSystemPowerDownWorker,
> +                        DelayedWorkQueue,
> +                        Fdo);
> +    } else {
> +        __FdoSetSystemPowerState(Fdo, SystemState);
> +
> +        IoCopyCurrentIrpStackLocationToNext(Irp); /* Irp has been pended */
> +        IoCallDriver(Fdo->LowerDeviceObject, Irp);
> +    }
>   }
>   
>   static FORCEINLINE NTSTATUS
> @@ -2383,33 +2616,44 @@ __FdoSetSystemPowerDown(
>   {
>       PIO_STACK_LOCATION  StackLocation;
>       SYSTEM_POWER_STATE  SystemState;
> -    DEVICE_POWER_STATE  DeviceState;
> +    POWER_STATE         PowerState;
>       NTSTATUS            status;
>   
>       StackLocation = IoGetCurrentIrpStackLocation(Irp);
> +    BUG_ON(StackLocation->DeviceObject != Fdo->Dx->DeviceObject);
>       SystemState = StackLocation->Parameters.Power.State.SystemState;
>   
>       ASSERT3U(SystemState, >,  __FdoGetSystemPowerState(Fdo));
>   
> -    DeviceState = Fdo->LowerDeviceCapabilities.DeviceState[SystemState];
> -
> -    FdoRequestSetDevicePower(Fdo, DeviceState);
> +    PowerState.DeviceState = Fdo->LowerDeviceCapabilities.DeviceState[SystemState];
>   
> -    Info("%s: %s -> %s\n",
> -         __FdoGetName(Fdo),
> -         PowerSystemStateName(__FdoGetSystemPowerState(Fdo)),
> -         PowerSystemStateName(SystemState));
> -
> -    if (SystemState >= PowerSystemHibernate &&
> -        __FdoGetSystemPowerState(Fdo) < PowerSystemHibernate) {
> -        __FdoSetSystemPowerState(Fdo, PowerSystemSleeping3);
> -        FdoS3ToS4(Fdo);
> +    if (SystemState >= PowerSystemShutdown) {
> +        /* No DIrp generation, no FDO Powerdown.
> +           Legacy shutdown just yanks system power.
> +           Best device state for S5 is advertised as D3,
> +           which is not *really* the case, but we can't increase it. */
> +        IoCopyCurrentIrpStackLocationToNext(Irp); /* Irp has been pended */
> +        status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
> +    } else {
> +        status = PoRequestPowerIrp(Fdo->LowerDeviceObject,
> +                                   IRP_MN_SET_POWER,
> +                                   PowerState,
> +                                   FdoRequestSetDevicePowerDownComplete,
> +                                   Irp,
> +                                   NULL);
> +        if (!NT_SUCCESS(status))
> +            goto fail1;
>       }
> +    /* IRP already marked pending - nothing more to do if awaiting callback. */
>   
> -    __FdoSetSystemPowerState(Fdo, SystemState);
> +    return status;
>   
> -    IoSkipCurrentIrpStackLocation(Irp);
> -    status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
> +fail1:
> +    /* In theory could change IRP status, but are not supposed to fail this IRP. */
> +    Error("fail1 - but continue IRP processing. (%08x)\n", status);
> +
> +    IoCopyCurrentIrpStackLocationToNext(Irp); /* Irp has been pended */
> +    IoCallDriver(Fdo->LowerDeviceObject, Irp);
>   
>       return status;
>   }
> @@ -2429,29 +2673,31 @@ __FdoSetSystemPower(
>       SystemState = StackLocation->Parameters.Power.State.SystemState;
>       PowerAction = StackLocation->Parameters.Power.ShutdownType;
>   
> +    /* IRP_MN_SET_POWER Setting system power, IRP's *must* be pended. */
> +    IoMarkIrpPending(Irp);
> +
>       Trace("====> (%s:%s)\n",
> -          PowerSystemStateName(SystemState),
> +          PowerSystemStateName(SystemState),
>             PowerActionName(PowerAction));
>   
> -    ASSERT3U(PowerAction, <, PowerActionShutdown);
> -
>       if (SystemState == __FdoGetSystemPowerState(Fdo)) {
> -        IoSkipCurrentIrpStackLocation(Irp);
> +        IoCopyCurrentIrpStackLocationToNext(Irp); /* Pended, copy not skip */
>           status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
>   
>           goto done;
>       }
>   
> -    status = (SystemState < __FdoGetSystemPowerState(Fdo)) ?
> +    status = SystemState < __FdoGetSystemPowerState(Fdo) ?
>                __FdoSetSystemPowerUp(Fdo, Irp) :
>                __FdoSetSystemPowerDown(Fdo, Irp);
>   
>   done:
>       Trace("<==== (%s:%s)(%08x)\n",
> -          PowerSystemStateName(SystemState),
> +          PowerSystemStateName(SystemState),
>             PowerActionName(PowerAction),
>             status);
> -    return status;
> +
> +    return STATUS_PENDING;
>   }
>   
>   static FORCEINLINE NTSTATUS
> @@ -2469,10 +2715,9 @@ __FdoQueryDevicePowerUp(
>   
>       ASSERT3U(DeviceState, <,  __FdoGetDevicePowerState(Fdo));
>   
> -    status = FdoForwardIrpSynchronously(Fdo, Irp);
> -
> -    Irp->IoStatus.Status = status;
> -    IoCompleteRequest(Irp, IO_NO_INCREMENT);
> +    /* N.B. Shutdown / hibernation failures caused by our device state could be vetoed here. */
> +    IoSkipCurrentIrpStackLocation(Irp);
> +    status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
>   
>       return status;
>   }
> @@ -2492,6 +2737,7 @@ __FdoQueryDevicePowerDown(
>   
>       ASSERT3U(DeviceState, >,  __FdoGetDevicePowerState(Fdo));
>   
> +    /* N.B. Shutdown / hibernation failures caused by our device state could be vetoed here. */
>       IoSkipCurrentIrpStackLocation(Irp);
>       status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
>   
> @@ -2514,11 +2760,9 @@ __FdoQueryDevicePower(
>       PowerAction = StackLocation->Parameters.Power.ShutdownType;
>   
>       Trace("====> (%s:%s)\n",
> -          PowerDeviceStateName(DeviceState),
> +          PowerDeviceStateName(DeviceState),
>             PowerActionName(PowerAction));
>   
> -    ASSERT3U(PowerAction, <, PowerActionShutdown);
> -
>       if (DeviceState == __FdoGetDevicePowerState(Fdo)) {
>           IoSkipCurrentIrpStackLocation(Irp);
>           status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
> @@ -2532,7 +2776,7 @@ __FdoQueryDevicePower(
>   
>   done:
>       Trace("<==== (%s:%s)(%08x)\n",
> -          PowerDeviceStateName(DeviceState),
> +          PowerDeviceStateName(DeviceState),
>             PowerActionName(PowerAction),
>             status);
>       return status;
> @@ -2540,8 +2784,8 @@ done:
>   
>   __drv_functionClass(REQUEST_POWER_COMPLETE)
>   __drv_sameIRQL
> -VOID
> -__FdoRequestQueryDevicePower(
> +static VOID
> +FdoRequestQueryDevicePowerUpComplete(
>       IN  PDEVICE_OBJECT      DeviceObject,
>       IN  UCHAR               MinorFunction,
>       IN  POWER_STATE         PowerState,
> @@ -2549,47 +2793,58 @@ __FdoRequestQueryDevicePower(
>       IN  PIO_STATUS_BLOCK    IoStatus
>       )
>   {
> -    PKEVENT                 Event = Context;
> +    PIRP                    Irp = (PIRP) Context;
>   
>       UNREFERENCED_PARAMETER(DeviceObject);
>       UNREFERENCED_PARAMETER(MinorFunction);
>       UNREFERENCED_PARAMETER(PowerState);
>   
> -    ASSERT(NT_SUCCESS(IoStatus->Status));
> +    if (!NT_SUCCESS(IoStatus->Status))
> +        Irp->IoStatus.Status = IoStatus->Status;
>   
> -    KeSetEvent(Event, IO_NO_INCREMENT, FALSE);
> +    IoCompleteRequest(Irp, IO_NO_INCREMENT);
>   }
>   
> -static VOID
> -FdoRequestQueryDevicePower(
> -    IN  PXENVIF_FDO         Fdo,
> -    IN  DEVICE_POWER_STATE  DeviceState
> +__drv_functionClass(IO_COMPLETION_ROUTINE)
> +__drv_sameIRQL
> +static NTSTATUS
> +FdoQuerySystemPowerUpComplete(
> +    IN  PDEVICE_OBJECT  DeviceObject,
> +    IN  PIRP            Irp,
> +    IN  PVOID           Context
>       )
>   {
> -    POWER_STATE             PowerState;
> -    KEVENT                  Event;
> -    NTSTATUS                status;
> -
> -    Trace("%s\n", PowerDeviceStateName(DeviceState));
> +    PXENVIF_FDO         Fdo = (PXENVIF_FDO) Context;
> +    PIO_STACK_LOCATION  StackLocation;
> +    SYSTEM_POWER_STATE  SystemState;
> +    POWER_STATE         PowerState;
> +    NTSTATUS            status;
>   
> -    ASSERT3U(KeGetCurrentIrql(), ==, PASSIVE_LEVEL);
> +    UNREFERENCED_PARAMETER(DeviceObject);
>   
> -    PowerState.DeviceState = DeviceState;
> -    KeInitializeEvent(&Event, NotificationEvent, FALSE);
> +    /* IRP marked as pending on dispatch path. */
> +    StackLocation = IoGetCurrentIrpStackLocation(Irp);
> +    SystemState = StackLocation->Parameters.Power.State.SystemState;
> +    PowerState.DeviceState = Fdo->LowerDeviceCapabilities.DeviceState[SystemState];
>   
>       status = PoRequestPowerIrp(Fdo->LowerDeviceObject,
>                                  IRP_MN_QUERY_POWER,
>                                  PowerState,
> -                               __FdoRequestQueryDevicePower,
> -                               &Event,
> +                               FdoRequestQueryDevicePowerUpComplete,
> +                               Irp,
>                                  NULL);
> -    ASSERT(NT_SUCCESS(status));
> +    if (!NT_SUCCESS(status))
> +        goto fail1;
>   
> -    (VOID) KeWaitForSingleObject(&Event,
> -                                 Executive,
> -                                 KernelMode,
> -                                 FALSE,
> -                                 NULL);
> +    /* Will later complete the IRP (which is marked pending, in 2nd level comp routine.)*/
> +    return STATUS_MORE_PROCESSING_REQUIRED;
> +
> +fail1:
> +    /* Irp marked as pending on dispatch path, so can change final IoStatus, pass to completion routine above. */
> +    Error("fail1 (%08x)\n", status);
> +    Irp->IoStatus.Status = status;
> +
> +    return STATUS_CONTINUE_COMPLETION;
>   }
>   
>   static FORCEINLINE NTSTATUS
> @@ -2598,30 +2853,61 @@ __FdoQuerySystemPowerUp(
>       IN  PIRP            Irp
>       )
>   {
> -
>       PIO_STACK_LOCATION  StackLocation;
>       SYSTEM_POWER_STATE  SystemState;
> -    DEVICE_POWER_STATE  DeviceState;
> -    NTSTATUS            status;
>   
>       StackLocation = IoGetCurrentIrpStackLocation(Irp);
>       SystemState = StackLocation->Parameters.Power.State.SystemState;
>   
>       ASSERT3U(SystemState, <,  __FdoGetSystemPowerState(Fdo));
>   
> -    status = FdoForwardIrpSynchronously(Fdo, Irp);
> -    if (!NT_SUCCESS(status))
> -        goto done;
> +    IoMarkIrpPending(Irp); /* Must mark IRP pending because we want to complete it *after* completion routine. */
> +    IoCopyCurrentIrpStackLocationToNext(Irp);
> +    IoSetCompletionRoutine(Irp,
> +                           FdoQuerySystemPowerUpComplete,
> +                           Fdo,
> +                           TRUE,
> +                           TRUE,
> +                           TRUE);
> +    (VOID) IoCallDriver(Fdo->LowerDeviceObject, Irp);
>   
> -    DeviceState = Fdo->LowerDeviceCapabilities.DeviceState[SystemState];
> +    return STATUS_PENDING;
> +}
>   
> -    FdoRequestQueryDevicePower(Fdo, DeviceState);
> +__drv_functionClass(REQUEST_POWER_COMPLETE)
> +__drv_sameIRQL
> +static VOID
> +FdoRequestQueryDevicePowerDownComplete(
> +    IN  PDEVICE_OBJECT      DeviceObject,
> +    IN  UCHAR               MinorFunction,
> +    IN  POWER_STATE         PowerState,
> +    IN  PVOID               Context,
> +    IN  PIO_STATUS_BLOCK    IoStatus
> +    )
> +{
> +    PIRP                    Irp = (PIRP) Context;
> +    PIO_STACK_LOCATION      StackLocation = IoGetCurrentIrpStackLocation(Irp);
> +    PDEVICE_OBJECT          UpperDeviceObject = StackLocation->DeviceObject;
> +    PXENVIF_DX              Dx = (PXENVIF_DX)UpperDeviceObject->DeviceExtension;
> +    PXENVIF_FDO             Fdo = Dx->Fdo;
>   
> -done:
> -    Irp->IoStatus.Status = status;
> -    IoCompleteRequest(Irp, IO_NO_INCREMENT);
> +    UNREFERENCED_PARAMETER(DeviceObject);
> +    UNREFERENCED_PARAMETER(MinorFunction);
> +    UNREFERENCED_PARAMETER(PowerState);
>   
> -    return status;
> +    if (!NT_SUCCESS(IoStatus->Status))
> +        goto fail1;
> +
> +    IoCopyCurrentIrpStackLocationToNext(Irp); /* Irp has been pended. */
> +    IoCallDriver(Fdo->LowerDeviceObject, Irp);
> +
> +    return;
> +
> +fail1:
> +    Error("fail1 (%08x)\n", IoStatus->Status);
> +
> +    Irp->IoStatus.Status = IoStatus->Status;
> +    IoCompleteRequest(Irp, IO_NO_INCREMENT);
>   }
>   
>   static FORCEINLINE NTSTATUS
> @@ -2632,7 +2918,7 @@ __FdoQuerySystemPowerDown(
>   {
>       PIO_STACK_LOCATION  StackLocation;
>       SYSTEM_POWER_STATE  SystemState;
> -    DEVICE_POWER_STATE  DeviceState;
> +    POWER_STATE         PowerState;
>       NTSTATUS            status;
>   
>       StackLocation = IoGetCurrentIrpStackLocation(Irp);
> @@ -2640,12 +2926,26 @@ __FdoQuerySystemPowerDown(
>   
>       ASSERT3U(SystemState, >,  __FdoGetSystemPowerState(Fdo));
>   
> -    DeviceState = Fdo->LowerDeviceCapabilities.DeviceState[SystemState];
> +    PowerState.DeviceState = Fdo->LowerDeviceCapabilities.DeviceState[SystemState];
>   
> -    FdoRequestQueryDevicePower(Fdo, DeviceState);
> +    status = PoRequestPowerIrp(Fdo->LowerDeviceObject,
> +                               IRP_MN_QUERY_POWER,
> +                               PowerState,
> +                               FdoRequestQueryDevicePowerDownComplete,
> +                               Irp,
> +                               NULL);
> +    if (!NT_SUCCESS(status))
> +        goto fail1;
>   
> -    IoSkipCurrentIrpStackLocation(Irp);
> -    status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
> +    IoMarkIrpPending(Irp);
> +
> +    return STATUS_PENDING;
> +
> +fail1:
> +    Error("fail1 (%08x)\n", status);
> +
> +    Irp->IoStatus.Status = status;
> +    IoCompleteRequest(Irp, IO_NO_INCREMENT);
>   
>       return status;
>   }
> @@ -2666,11 +2966,9 @@ __FdoQuerySystemPower(
>       PowerAction = StackLocation->Parameters.Power.ShutdownType;
>   
>       Trace("====> (%s:%s)\n",
> -          PowerSystemStateName(SystemState),
> +          PowerSystemStateName(SystemState),
>             PowerActionName(PowerAction));
>   
> -    ASSERT3U(PowerAction, <, PowerActionShutdown);
> -
>       if (SystemState == __FdoGetSystemPowerState(Fdo)) {
>           IoSkipCurrentIrpStackLocation(Irp);
>           status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
> @@ -2684,183 +2982,91 @@ __FdoQuerySystemPower(
>   
>   done:
>       Trace("<==== (%s:%s)(%08x)\n",
> -          PowerSystemStateName(SystemState),
> +          PowerSystemStateName(SystemState),
>             PowerActionName(PowerAction),
>             status);
>   
>       return status;
>   }
>   
> -static NTSTATUS
> -FdoDevicePower(
> -    IN  PXENVIF_THREAD  Self,
> -    IN  PVOID           Context
> +static FORCEINLINE NTSTATUS
> +__FdoDevicePower(
> +    IN  PXENVIF_FDO     Fdo,
> +    IN  PIRP            Irp
>       )
>   {
> -    PXENVIF_FDO         Fdo = Context;
> -    PKEVENT             Event;
> -
> -    Event = ThreadGetEvent(Self);
> -
> -    for (;;) {
> -        PIRP                Irp;
> -        PIO_STACK_LOCATION  StackLocation;
> -        UCHAR               MinorFunction;
> -
> -        if (Fdo->DevicePowerIrp == NULL) {
> -            (VOID) KeWaitForSingleObject(Event,
> -                                         Executive,
> -                                         KernelMode,
> -                                         FALSE,
> -                                         NULL);
> -            KeClearEvent(Event);
> -        }
> -
> -        if (ThreadIsAlerted(Self))
> -            break;
> -
> -        Irp = Fdo->DevicePowerIrp;
> -
> -        if (Irp == NULL)
> -            continue;
> -
> -        Fdo->DevicePowerIrp = NULL;
> -        KeMemoryBarrier();
> +    PIO_STACK_LOCATION  StackLocation;
> +    NTSTATUS            status;
>   
> -        StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -        MinorFunction = StackLocation->MinorFunction;
> +    StackLocation = IoGetCurrentIrpStackLocation(Irp);
>   
> -        switch (StackLocation->MinorFunction) {
> -        case IRP_MN_SET_POWER:
> -            (VOID) __FdoSetDevicePower(Fdo, Irp);
> -            break;
> +    switch (StackLocation->MinorFunction) {
> +    case IRP_MN_SET_POWER:
> +        status = __FdoSetDevicePower(Fdo, Irp);
> +        break;
>   
> -        case IRP_MN_QUERY_POWER:
> -            (VOID) __FdoQueryDevicePower(Fdo, Irp);
> -            break;
> +    case IRP_MN_QUERY_POWER:
> +        status = __FdoQueryDevicePower(Fdo, Irp);
> +        break;
>   
> -        default:
> -            ASSERT(FALSE);
> -            break;
> -        }
> +    default:
> +        IoSkipCurrentIrpStackLocation(Irp);
> +        status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
> +        break;
>       }
>   
> -    return STATUS_SUCCESS;
> +    return status;
>   }
>   
> -static NTSTATUS
> -FdoSystemPower(
> -    IN  PXENVIF_THREAD  Self,
> -    IN  PVOID           Context
> +static FORCEINLINE NTSTATUS
> +__FdoSystemPower(
> +    IN  PXENVIF_FDO     Fdo,
> +    IN  PIRP            Irp
>       )
>   {
> -    PXENVIF_FDO         Fdo = Context;
> -    PKEVENT             Event;
> -
> -    Event = ThreadGetEvent(Self);
> -
> -    for (;;) {
> -        PIRP                Irp;
> -        PIO_STACK_LOCATION  StackLocation;
> -        UCHAR               MinorFunction;
> -
> -        if (Fdo->SystemPowerIrp == NULL) {
> -            (VOID) KeWaitForSingleObject(Event,
> -                                         Executive,
> -                                         KernelMode,
> -                                         FALSE,
> -                                         NULL);
> -            KeClearEvent(Event);
> -        }
> -
> -        if (ThreadIsAlerted(Self))
> -            break;
> -
> -        Irp = Fdo->SystemPowerIrp;
> -
> -        if (Irp == NULL)
> -            continue;
> -
> -        Fdo->SystemPowerIrp = NULL;
> -        KeMemoryBarrier();
> +    PIO_STACK_LOCATION  StackLocation;
> +    NTSTATUS            status;
>   
> -        StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -        MinorFunction = StackLocation->MinorFunction;
> +    StackLocation = IoGetCurrentIrpStackLocation(Irp);
>   
> -        switch (StackLocation->MinorFunction) {
> -        case IRP_MN_SET_POWER:
> -            (VOID) __FdoSetSystemPower(Fdo, Irp);
> -            break;
> +    switch (StackLocation->MinorFunction) {
> +    case IRP_MN_SET_POWER:
> +        status = __FdoSetSystemPower(Fdo, Irp);
> +        break;
>   
> -        case IRP_MN_QUERY_POWER:
> -            (VOID) __FdoQuerySystemPower(Fdo, Irp);
> -            break;
> +    case IRP_MN_QUERY_POWER:
> +        status = __FdoQuerySystemPower(Fdo, Irp);
> +        break;
>   
> -        default:
> -            ASSERT(FALSE);
> -            break;
> -        }
> +    default:
> +        IoSkipCurrentIrpStackLocation(Irp);
> +        status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
> +        break;
>       }
>   
> -    return STATUS_SUCCESS;
> +    return status;
>   }
>   
>   static DECLSPEC_NOINLINE NTSTATUS
>   FdoDispatchPower(
> -    IN  PXENVIF_FDO     Fdo,
> +    IN  PXENVIF_FDO   Fdo,
>       IN  PIRP            Irp
>       )
>   {
>       PIO_STACK_LOCATION  StackLocation;
> -    UCHAR               MinorFunction;
>       POWER_STATE_TYPE    PowerType;
> -    POWER_ACTION        PowerAction;
>       NTSTATUS            status;
>   
>       StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -    MinorFunction = StackLocation->MinorFunction;
> -
> -    if (MinorFunction != IRP_MN_QUERY_POWER &&
> -        MinorFunction != IRP_MN_SET_POWER) {
> -        IoSkipCurrentIrpStackLocation(Irp);
> -        status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
> -
> -        goto done;
> -    }
> -
>       PowerType = StackLocation->Parameters.Power.Type;
> -    PowerAction = StackLocation->Parameters.Power.ShutdownType;
> -
> -    if (PowerAction >= PowerActionShutdown) {
> -        IoSkipCurrentIrpStackLocation(Irp);
> -        status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
> -
> -        goto done;
> -    }
>   
>       switch (PowerType) {
>       case DevicePowerState:
> -        IoMarkIrpPending(Irp);
> -
> -        ASSERT3P(Fdo->DevicePowerIrp, ==, NULL);
> -        Fdo->DevicePowerIrp = Irp;
> -        KeMemoryBarrier();
> -
> -        ThreadWake(Fdo->DevicePowerThread);
> -
> -        status = STATUS_PENDING;
> +        status = __FdoDevicePower(Fdo, Irp);
>           break;
>   
>       case SystemPowerState:
> -        IoMarkIrpPending(Irp);
> -
> -        ASSERT3P(Fdo->SystemPowerIrp, ==, NULL);
> -        Fdo->SystemPowerIrp = Irp;
> -        KeMemoryBarrier();
> -
> -        ThreadWake(Fdo->SystemPowerThread);
> -
> -        status = STATUS_PENDING;
> +        status = __FdoSystemPower(Fdo, Irp);
>           break;
>   
>       default:
> @@ -2869,7 +3075,6 @@ FdoDispatchPower(
>           break;
>       }
>   
> -done:
>       return status;
>   }
>   
> @@ -3063,12 +3268,12 @@ FdoCreate(
>       Fdo->LowerDeviceObject = IoAttachDeviceToDeviceStack(FunctionDeviceObject,
>                                                            PhysicalDeviceObject);
>   
> -    status = ThreadCreate(FdoSystemPower, Fdo, &Fdo->SystemPowerThread);
> -    if (!NT_SUCCESS(status))
> +    Fdo->SystemPowerWorkItem = IoAllocateWorkItem(PhysicalDeviceObject);
> +    if (Fdo->SystemPowerWorkItem == NULL)
>           goto fail3;
>   
> -    status = ThreadCreate(FdoDevicePower, Fdo, &Fdo->DevicePowerThread);
> -    if (!NT_SUCCESS(status))
> +    Fdo->DevicePowerWorkItem = IoAllocateWorkItem(PhysicalDeviceObject);
> +    if (Fdo->DevicePowerWorkItem == NULL)
>           goto fail4;
>   
>       status = __FdoAcquireLowerBusInterface(Fdo);
> @@ -3226,16 +3431,14 @@ fail6:
>   fail5:
>       Error("fail5\n");
>   
> -    ThreadAlert(Fdo->DevicePowerThread);
> -    ThreadJoin(Fdo->DevicePowerThread);
> -    Fdo->DevicePowerThread = NULL;
> -
> +    IoFreeWorkItem(Fdo->DevicePowerWorkItem);
> +    Fdo->DevicePowerWorkItem = NULL;
> +
>   fail4:
>       Error("fail4\n");
>   
> -    ThreadAlert(Fdo->SystemPowerThread);
> -    ThreadJoin(Fdo->SystemPowerThread);
> -    Fdo->SystemPowerThread = NULL;
> +    IoFreeWorkItem(Fdo->SystemPowerWorkItem);
> +    Fdo->SystemPowerWorkItem = NULL;
>       
>   fail3:
>       Error("fail3\n");
> @@ -3311,13 +3514,11 @@ FdoDestroy(
>   
>       __FdoReleaseLowerBusInterface(Fdo);
>   
> -    ThreadAlert(Fdo->DevicePowerThread);
> -    ThreadJoin(Fdo->DevicePowerThread);
> -    Fdo->DevicePowerThread = NULL;
> +    IoFreeWorkItem(Fdo->DevicePowerWorkItem);
> +    Fdo->DevicePowerWorkItem = NULL;
>   
> -    ThreadAlert(Fdo->SystemPowerThread);
> -    ThreadJoin(Fdo->SystemPowerThread);
> -    Fdo->SystemPowerThread = NULL;
> +    IoFreeWorkItem(Fdo->SystemPowerWorkItem);
> +    Fdo->SystemPowerWorkItem = NULL;
>   
>       IoDetachDevice(Fdo->LowerDeviceObject);
>   
> diff --git a/src/xenvif/pdo.c b/src/xenvif/pdo.c
> index 0cef2d5..2979f53 100644
> --- a/src/xenvif/pdo.c
> +++ b/src/xenvif/pdo.c
> @@ -68,9 +68,9 @@
>   struct _XENVIF_PDO {
>       PXENVIF_DX                  Dx;
>   
> -    PXENVIF_THREAD              SystemPowerThread;
> +    PIO_WORKITEM                SystemPowerWorkItem;
>       PIRP                        SystemPowerIrp;
> -    PXENVIF_THREAD              DevicePowerThread;
> +    PIO_WORKITEM                DevicePowerWorkItem;
>       PIRP                        DevicePowerIrp;
>   
>       PXENVIF_FDO                 Fdo;
> @@ -2365,28 +2365,31 @@ PdoDispatchPnp(
>       return status;
>   }
>   
> -static FORCEINLINE NTSTATUS
> -__PdoSetDevicePower(
> -    IN  PXENVIF_PDO     Pdo,
> -    IN  PIRP            Irp
> +__drv_functionClass(IO_WORKITEM_ROUTINE)
> +__drv_sameIRQL
> +static VOID
> +PdoDevicePowerWorker(
> +    IN  PDEVICE_OBJECT  DeviceObject,
> +    IN  PVOID           Context
>       )
>   {
> +    PXENVIF_PDO         Pdo = (PXENVIF_PDO) Context;
> +    PIRP                Irp;
>       PIO_STACK_LOCATION  StackLocation;
>       DEVICE_POWER_STATE  DeviceState;
>       POWER_ACTION        PowerAction;
>       NTSTATUS            status;
>   
> +    UNREFERENCED_PARAMETER(DeviceObject);
> +
> +    Irp = InterlockedExchangePointer(&Pdo->DevicePowerIrp, NULL);
> +    ASSERT(Irp != NULL);
> +
>       StackLocation = IoGetCurrentIrpStackLocation(Irp);
>       DeviceState = StackLocation->Parameters.Power.State.DeviceState;
>       PowerAction = StackLocation->Parameters.Power.ShutdownType;
>   
> -    Trace("====> (%s) (%s:%s)\n",
> -          __PdoGetName(Pdo),
> -          PowerDeviceStateName(DeviceState),
> -          PowerActionName(PowerAction));
> -
> -    ASSERT3U(PowerAction, <, PowerActionShutdown);
> -
> +    status = STATUS_SUCCESS;
>       if (__PdoGetDevicePowerState(Pdo) > DeviceState) {
>           Trace("%s: POWERING UP: %s -> %s\n",
>                 __PdoGetName(Pdo),
> @@ -2395,7 +2398,6 @@ __PdoSetDevicePower(
>   
>           ASSERT3U(DeviceState, ==, PowerDeviceD0);
>           status = PdoD3ToD0(Pdo);
> -        ASSERT(NT_SUCCESS(status));
>       } else if (__PdoGetDevicePowerState(Pdo) < DeviceState) {
>           Trace("%s: POWERING DOWN: %s -> %s\n",
>                 __PdoGetName(Pdo),
> @@ -2406,77 +2408,71 @@ __PdoSetDevicePower(
>           PdoD0ToD3(Pdo);
>       }
>   
> +    /* Cannot fail the IRP at this point, keep going. */
>       Irp->IoStatus.Status = STATUS_SUCCESS;
>       IoCompleteRequest(Irp, IO_NO_INCREMENT);
>   
> -    Trace("<==== (%s:%s)\n",
> +    Trace("<==== (%s:%s)(%08x)\n",
>             PowerDeviceStateName(DeviceState),
> -          PowerActionName(PowerAction));
> -
> -    return STATUS_SUCCESS;
> +          PowerActionName(PowerAction),
> +          status);
>   }
>   
> -static NTSTATUS
> -PdoDevicePower(
> -    IN  PXENVIF_THREAD  Self,
> -    IN  PVOID           Context
> +static FORCEINLINE NTSTATUS
> +__PdoSetDevicePower(
> +    IN  PXENVIF_PDO     Pdo,
> +    IN  PIRP            Irp
>       )
>   {
> -    PXENVIF_PDO         Pdo = Context;
> -    PKEVENT             Event;
> -
> -    Event = ThreadGetEvent(Self);
> -
> -    for (;;) {
> -        PIRP    Irp;
> -
> -        if (Pdo->DevicePowerIrp == NULL) {
> -            (VOID) KeWaitForSingleObject(Event,
> -                                         Executive,
> -                                         KernelMode,
> -                                         FALSE,
> -                                         NULL);
> -            KeClearEvent(Event);
> -        }
> +    PIO_STACK_LOCATION  StackLocation;
> +    DEVICE_POWER_STATE  DeviceState;
> +    POWER_ACTION        PowerAction;
> +    PVOID               Exchange;
>   
> -        if (ThreadIsAlerted(Self))
> -            break;
> +    StackLocation = IoGetCurrentIrpStackLocation(Irp);
> +    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
> +    PowerAction = StackLocation->Parameters.Power.ShutdownType;
>   
> -        Irp = Pdo->DevicePowerIrp;
> +    Trace("====> (%s:%s)\n",
> +          PowerDeviceStateName(DeviceState),
> +          PowerActionName(PowerAction));
>   
> -        if (Irp == NULL)
> -            continue;
> +    IoMarkIrpPending(Irp);
>   
> -        Pdo->DevicePowerIrp = NULL;
> -        KeMemoryBarrier();
> +    Exchange = InterlockedExchangePointer(&Pdo->DevicePowerIrp, Irp);
> +    ASSERT(Exchange == NULL);
>   
> -        (VOID) __PdoSetDevicePower(Pdo, Irp);
> -    }
> +    IoQueueWorkItem(Pdo->DevicePowerWorkItem,
> +                    PdoDevicePowerWorker,
> +                    DelayedWorkQueue,
> +                    Pdo);
>   
> -    return STATUS_SUCCESS;
> +    return STATUS_PENDING;
>   }
>   
> -static FORCEINLINE NTSTATUS
> -__PdoSetSystemPower(
> -    IN  PXENVIF_PDO         Pdo,
> -    IN  PIRP                Irp
> +__drv_functionClass(IO_WORKITEM_ROUTINE)
> +__drv_sameIRQL
> +static VOID
> +PdoSystemPowerWorker(
> +    IN  PDEVICE_OBJECT  DeviceObject,
> +    IN  PVOID           Context
>       )
>   {
> -    PIO_STACK_LOCATION      StackLocation;
> -    SYSTEM_POWER_STATE      SystemState;
> -    POWER_ACTION            PowerAction;
> +    PXENVIF_PDO         Pdo = (PXENVIF_PDO) Context;
> +    PIRP                Irp;
> +    PIO_STACK_LOCATION  StackLocation;
> +    SYSTEM_POWER_STATE  SystemState;
> +    POWER_ACTION        PowerAction;
> +
> +    UNREFERENCED_PARAMETER(DeviceObject);
> +
> +    Irp = InterlockedExchangePointer(&Pdo->SystemPowerIrp, NULL);
> +    ASSERT(Irp != NULL);
>   
>       StackLocation = IoGetCurrentIrpStackLocation(Irp);
>       SystemState = StackLocation->Parameters.Power.State.SystemState;
>       PowerAction = StackLocation->Parameters.Power.ShutdownType;
>   
> -    Trace("====> (%s) (%s:%s)\n",
> -          __PdoGetName(Pdo),
> -          PowerSystemStateName(SystemState),
> -          PowerActionName(PowerAction));
> -
> -    ASSERT3U(PowerAction, <, PowerActionShutdown);
> -
>       if (__PdoGetSystemPowerState(Pdo) > SystemState) {
>           if (SystemState < PowerSystemHibernate &&
>               __PdoGetSystemPowerState(Pdo) >= PowerSystemHibernate) {
> @@ -2488,6 +2484,7 @@ __PdoSetSystemPower(
>                 __PdoGetName(Pdo),
>                 PowerSystemStateName(__PdoGetSystemPowerState(Pdo)),
>                 PowerSystemStateName(SystemState));
> +
>       } else if (__PdoGetSystemPowerState(Pdo) < SystemState) {
>           Trace("%s: POWERING DOWN: %s -> %s\n",
>                 __PdoGetName(Pdo),
> @@ -2509,97 +2506,60 @@ __PdoSetSystemPower(
>       Trace("<==== (%s:%s)\n",
>             PowerSystemStateName(SystemState),
>             PowerActionName(PowerAction));
> -
> -    return STATUS_SUCCESS;
>   }
>   
> -static NTSTATUS
> -PdoSystemPower(
> -    IN  PXENVIF_THREAD  Self,
> -    IN  PVOID           Context
> +static FORCEINLINE NTSTATUS
> +__PdoSetSystemPower(
> +    IN  PXENVIF_PDO     Pdo,
> +    IN  PIRP            Irp
>       )
>   {
> -    PXENVIF_PDO         Pdo = Context;
> -    PKEVENT             Event;
> -
> -    Event = ThreadGetEvent(Self);
> -
> -    for (;;) {
> -        PIRP    Irp;
> -
> -        if (Pdo->SystemPowerIrp == NULL) {
> -            (VOID) KeWaitForSingleObject(Event,
> -                                         Executive,
> -                                         KernelMode,
> -                                         FALSE,
> -                                         NULL);
> -            KeClearEvent(Event);
> -        }
> +    PIO_STACK_LOCATION  StackLocation;
> +    SYSTEM_POWER_STATE  SystemState;
> +    POWER_ACTION        PowerAction;
> +    PVOID               Exchange;
>   
> -        if (ThreadIsAlerted(Self))
> -            break;
> +    StackLocation = IoGetCurrentIrpStackLocation(Irp);
> +    SystemState = StackLocation->Parameters.Power.State.SystemState;
> +    PowerAction = StackLocation->Parameters.Power.ShutdownType;
>   
> -        Irp = Pdo->SystemPowerIrp;
> +    Trace("====> (%s:%s)\n",
> +          PowerSystemStateName(SystemState),
> +          PowerActionName(PowerAction));
>   
> -        if (Irp == NULL)
> -            continue;
> +    IoMarkIrpPending(Irp);
>   
> -        Pdo->SystemPowerIrp = NULL;
> -        KeMemoryBarrier();
> +    Exchange = InterlockedExchangePointer(&Pdo->SystemPowerIrp, Irp);
> +    ASSERT(Exchange == NULL);
>   
> -        (VOID) __PdoSetSystemPower(Pdo, Irp);
> -    }
> +    IoQueueWorkItem(Pdo->SystemPowerWorkItem,
> +                    PdoSystemPowerWorker,
> +                    DelayedWorkQueue,
> +                    Pdo);
>   
> -    return STATUS_SUCCESS;
> +    return STATUS_PENDING;
>   }
>   
> -static DECLSPEC_NOINLINE NTSTATUS
> -PdoSetPower(
> +static FORCEINLINE NTSTATUS
> +__PdoSetPower(
>       IN  PXENVIF_PDO     Pdo,
>       IN  PIRP            Irp
>       )
>   {
>       PIO_STACK_LOCATION  StackLocation;
>       POWER_STATE_TYPE    PowerType;
> -    POWER_ACTION        PowerAction;
>       NTSTATUS            status;
>       
>       StackLocation = IoGetCurrentIrpStackLocation(Irp);
>       PowerType = StackLocation->Parameters.Power.Type;
> -    PowerAction = StackLocation->Parameters.Power.ShutdownType;
> -
> -    if (PowerAction >= PowerActionShutdown) {
> -        Irp->IoStatus.Status = STATUS_SUCCESS;
> -
> -        status = Irp->IoStatus.Status;
> -        IoCompleteRequest(Irp, IO_NO_INCREMENT);
> -
> -        goto done;
> -    }
>   
>       switch (PowerType) {
>       case DevicePowerState:
> -        IoMarkIrpPending(Irp);
> -
> -        ASSERT3P(Pdo->DevicePowerIrp, ==, NULL);
> -        Pdo->DevicePowerIrp = Irp;
> -        KeMemoryBarrier();
> -
> -        ThreadWake(Pdo->DevicePowerThread);
> -
> -        status = STATUS_PENDING;
> +        status = __PdoSetDevicePower(Pdo, Irp);
>           break;
>   
>       case SystemPowerState:
> -        IoMarkIrpPending(Irp);
> -
> -        ASSERT3P(Pdo->SystemPowerIrp, ==, NULL);
> -        Pdo->SystemPowerIrp = Irp;
> -        KeMemoryBarrier();
> -
> -        ThreadWake(Pdo->SystemPowerThread);
> -
> -        status = STATUS_PENDING;
> +        status = __PdoSetSystemPower(Pdo, Irp);
>           break;
>   
>       default:
> @@ -2608,48 +2568,41 @@ PdoSetPower(
>           break;
>       }
>   
> -done:
>       return status;
>   }
>   
> -static DECLSPEC_NOINLINE NTSTATUS
> -PdoQueryPower(
> -    IN  PXENVIF_PDO     Pdo,
> -    IN  PIRP            Irp
> +static FORCEINLINE NTSTATUS
> +__PdoQueryPower(
> +    IN  PXENVIF_PDO Pdo,
> +    IN  PIRP        Irp
>       )
>   {
> -    NTSTATUS            status;
> -
>       UNREFERENCED_PARAMETER(Pdo);
>   
>       Irp->IoStatus.Status = STATUS_SUCCESS;
> -
> -    status = Irp->IoStatus.Status;
>       IoCompleteRequest(Irp, IO_NO_INCREMENT);
> -
> -    return status;
> +
> +    return STATUS_SUCCESS;
>   }
>   
> -static DECLSPEC_NOINLINE NTSTATUS
> +static NTSTATUS
>   PdoDispatchPower(
>       IN  PXENVIF_PDO     Pdo,
>       IN  PIRP            Irp
>       )
>   {
>       PIO_STACK_LOCATION  StackLocation;
> -    UCHAR               MinorFunction;
>       NTSTATUS            status;
>   
>       StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -    MinorFunction = StackLocation->MinorFunction;
>   
>       switch (StackLocation->MinorFunction) {
>       case IRP_MN_SET_POWER:
> -        status = PdoSetPower(Pdo, Irp);
> +        status = __PdoSetPower(Pdo, Irp);
>           break;
>   
>       case IRP_MN_QUERY_POWER:
> -        status = PdoQueryPower(Pdo, Irp);
> +        status = __PdoQueryPower(Pdo, Irp);
>           break;
>   
>       default:
> @@ -2762,12 +2715,12 @@ PdoCreate(
>       Pdo->Dx = Dx;
>       Pdo->Fdo = Fdo;
>   
> -    status = ThreadCreate(PdoSystemPower, Pdo, &Pdo->SystemPowerThread);
> -    if (!NT_SUCCESS(status))
> +    Pdo->SystemPowerWorkItem = IoAllocateWorkItem(PhysicalDeviceObject);
> +    if (Pdo->SystemPowerWorkItem == NULL)
>           goto fail3;
>   
> -    status = ThreadCreate(PdoDevicePower, Pdo, &Pdo->DevicePowerThread);
> -    if (!NT_SUCCESS(status))
> +    Pdo->DevicePowerWorkItem = IoAllocateWorkItem(PhysicalDeviceObject);
> +    if (Pdo->DevicePowerWorkItem == NULL)
>           goto fail4;
>   
>       __PdoSetName(Pdo, Number);
> @@ -2860,16 +2813,14 @@ fail6:
>   fail5:
>       Error("fail5\n");
>   
> -    ThreadAlert(Pdo->DevicePowerThread);
> -    ThreadJoin(Pdo->DevicePowerThread);
> -    Pdo->DevicePowerThread = NULL;
> +    IoFreeWorkItem(Pdo->DevicePowerWorkItem);
> +    Pdo->DevicePowerWorkItem = NULL;
>   
>   fail4:
>       Error("fail4\n");
>   
> -    ThreadAlert(Pdo->SystemPowerThread);
> -    ThreadJoin(Pdo->SystemPowerThread);
> -    Pdo->SystemPowerThread = NULL;
> +    IoFreeWorkItem(Pdo->SystemPowerWorkItem);
> +    Pdo->SystemPowerWorkItem = NULL;
>   
>   fail3:
>       Error("fail3\n");
> @@ -2939,13 +2890,11 @@ PdoDestroy(
>   
>       __PdoClearPermanentAddress(Pdo);
>   
> -    ThreadAlert(Pdo->DevicePowerThread);
> -    ThreadJoin(Pdo->DevicePowerThread);
> -    Pdo->DevicePowerThread = NULL;
> +    IoFreeWorkItem(Pdo->DevicePowerWorkItem);
> +    Pdo->DevicePowerWorkItem = NULL;
>   
> -    ThreadAlert(Pdo->SystemPowerThread);
> -    ThreadJoin(Pdo->SystemPowerThread);
> -    Pdo->SystemPowerThread = NULL;
> +    IoFreeWorkItem(Pdo->SystemPowerWorkItem);
> +    Pdo->SystemPowerWorkItem = NULL;
>   
>       Pdo->Fdo = NULL;
>       Pdo->Dx = NULL;



From win-pv-devel-bounces@lists.xenproject.org Thu Mar 14 10:32:08 2024
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Thu, 14 Mar 2024 10:32:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.693121.1080866 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rkiNc-0005sl-OC; Thu, 14 Mar 2024 10:32:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 693121.1080866; Thu, 14 Mar 2024 10:32:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rkiNc-0005sb-LN; Thu, 14 Mar 2024 10:32:08 +0000
Received: by outflank-mailman (input) for mailman id 693121;
 Thu, 14 Mar 2024 10:32:07 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xOC7=KU=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1rkiNb-0005sV-C9
 for win-pv-devel@lists.xenproject.org; Thu, 14 Mar 2024 10:32:07 +0000
Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com
 [2a00:1450:4864:20::336])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 19ac91c5-e1ee-11ee-a1ee-f123f15fe8a2;
 Thu, 14 Mar 2024 11:32:05 +0100 (CET)
Received: by mail-wm1-x336.google.com with SMTP id
 5b1f17b1804b1-413e93b10bfso5004345e9.1
 for <win-pv-devel@lists.xenproject.org>; Thu, 14 Mar 2024 03:32:04 -0700 (PDT)
Received: from [10.95.157.67] (54-240-197-239.amazon.com. [54.240.197.239])
 by smtp.gmail.com with ESMTPSA id
 o18-20020a05600c4fd200b004132f9cf053sm5155658wmq.33.2024.03.14.03.32.03
 for <win-pv-devel@lists.xenproject.org>
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 14 Mar 2024 03:32:03 -0700 (PDT)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 19ac91c5-e1ee-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1710412324; x=1711017124; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:organization:reply-to
         :references:to:content-language:subject:user-agent:mime-version:date
         :message-id:from:from:to:cc:subject:date:message-id:reply-to;
        bh=VTv4y1a6CeOLeJZBGJtM5A3njjA917AnpBY3x96GgOg=;
        b=hVp45+hLfZcyxaB8BaRBZ3+meGow0MHFJTPqTYni3gJcmxE+KG2iPwEa81oGYexTA6
         HKIzuLTdRyFNZrj4IXhAeh+E4GYEk1bbTCMdym/tqlEBscuvdoeaMr+nUf4nEtc+cLd+
         4JL5/iVo3dpAUBrcnun11xkJYOd1d0cuyS6Q0j/Eu5T0WxqVkbQTHdyVoIfVS/NPkarU
         RZZne2w11975OzZo3FwfUgqmewvlFImk0uooGoPJnBc8rq9U0xiMyhVsZuT7Zp55qamX
         50E/2ZIWohBT4fLIigl0m6Ve+5BJwINQbJ06fpFMQn7Vk383lPXk0JrZkVgTBdruDC7/
         0yUg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1710412324; x=1711017124;
        h=content-transfer-encoding:in-reply-to:organization:reply-to
         :references:to:content-language:subject:user-agent:mime-version:date
         :message-id:from:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=VTv4y1a6CeOLeJZBGJtM5A3njjA917AnpBY3x96GgOg=;
        b=SoGZj1zMpke1dWToKo+SJbNu+VSkOlq6R6WRxZt6PMbpsFFb43eGjMcfSONEVjFVy1
         wLopvN6JsWmvXtABNWn8LGEz68svUd5RdpWYYFSufAMcdK6SGOIAXmbJjcLvqiw6j1JJ
         nmGMneiciScrE3jlqAU2RNO7i0XW+t1XY3z3mu7fR0NdLgUBEdyQFrM1tyduOHYofjNz
         u/MZkdi5CLq2Q862mf5LS+3zOpEJz/j2LeyIGQ/cuxyyrJyFToxLml7i4p2l5EVpfdHQ
         27EMOh/cy9aJyP0boKEeGaUGrw0plLotJl7ph+vT+jXnb0EKYvr+Bk4nSsXVJFX/ye0m
         wGKQ==
X-Gm-Message-State: AOJu0YyFHzxIQvwYMK02xWoeEBpSkIDnk7YScobV/GiwMTlJH9rSxGpA
	otpWBPUW3/46FB0o76j1Sol1fWq0uoWaN+d6X9GyI7qFvKWtXcNIVdVw2AhuxOY=
X-Google-Smtp-Source: AGHT+IE2OIaXgSYxl57QJl1LtnBSlngkNI6sQrz7O8tkQ21SpHy6GEHT9Soe3ltiQJXPyBR6rJ/eOA==
X-Received: by 2002:a05:600c:4746:b0:413:2ff9:bb04 with SMTP id w6-20020a05600c474600b004132ff9bb04mr1443703wmo.11.1710412323989;
        Thu, 14 Mar 2024 03:32:03 -0700 (PDT)
From: Paul Durrant <xadimgnik@gmail.com>
X-Google-Original-From: Paul Durrant <paul@xen.org>
Message-ID: <b574dbb2-c28a-4f64-ac5e-c7d197f46084@xen.org>
Date: Thu, 14 Mar 2024 10:31:57 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/2] Adjust Packet's MDL when stripping ETHERNET_TAG from
 headers
Content-Language: en-US
To: win-pv-devel@lists.xenproject.org
References: <20240305153331.1555-1-owen.smith@cloud.com>
Reply-To: paul@xen.org
Organization: Xen Project
In-Reply-To: <20240305153331.1555-1-owen.smith@cloud.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 05/03/2024 15:33, Owen Smith wrote:
> Tagged headers have the ETHERNET_TAG stripped to convert to
> ETHERNET_UNTAGGED_HEADERs. As part of the fix up, the offsets for each
> header was adjusted, and the underlying MDL's MappedSystemVa was adjusted.
> However, the ByteCount and ByteOffset were not adjusted, resulting in
> instances of data corruption when MappedSystemVa != StartVa + ByteOffset.
> 
> It is possible to configure the default gateway to respond to ARP packets
> with TPID wrappings. Openvswitch will not strip this header since this commit:
> 
>      commit f0fb825a3785320430686834741c718ff4f8ebf4
>      Author: Eric Garver <e@erig.me>
>      Date:   Wed Mar 1 17:47:59 2017 -0500
> 
>          Add support for 802.1ad (QinQ tunneling)
> 
> Which means Windows was receiving corrupted ARP responses, leading to an
> inability to access the network correctly. This would usually show as
> Windows indicating that no internet access was available, or being unable
> it access other VMs or other resources beyond the default gateway.
> 
> Signed-off-by: Owen Smith <owen.smith@cloud.com>

Reviewed-by: Paul Durrant <paul@xen.org>

> ---
>   src/xenvif/receiver.c | 7 ++++---
>   1 file changed, 4 insertions(+), 3 deletions(-)
> 
> diff --git a/src/xenvif/receiver.c b/src/xenvif/receiver.c
> index 0ce5b67..37e2c60 100644
> --- a/src/xenvif/receiver.c
> +++ b/src/xenvif/receiver.c
> @@ -442,10 +442,11 @@ ReceiverRingProcessTag(
>                     Offset);
>   
>       // Fix up the packet information
> -    BaseVa += sizeof (ETHERNET_TAG);
> +    Packet->Mdl.MappedSystemVa = (PUCHAR)Packet->Mdl.MappedSystemVa + sizeof(ETHERNET_TAG);
> +    Packet->Mdl.ByteOffset += sizeof(ETHERNET_TAG);
> +    Packet->Mdl.ByteCount -= sizeof(ETHERNET_TAG);
>   
> -    BaseVa -= Packet->Offset;
> -    Packet->Mdl.MappedSystemVa = BaseVa;
> +    ASSERT3P((PUCHAR)Packet->Mdl.StartVa + Packet->Mdl.ByteOffset, ==, Packet->Mdl.MappedSystemVa);
>   
>       Packet->Length -= sizeof (ETHERNET_TAG);
>   



From win-pv-devel-bounces@lists.xenproject.org Thu Mar 14 10:33:14 2024
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Thu, 14 Mar 2024 10:33:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.693131.1080870 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rkiOg-0005zD-Ru; Thu, 14 Mar 2024 10:33:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 693131.1080870; Thu, 14 Mar 2024 10:33:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rkiOg-0005z6-P9; Thu, 14 Mar 2024 10:33:14 +0000
Received: by outflank-mailman (input) for mailman id 693131;
 Thu, 14 Mar 2024 10:33:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xOC7=KU=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1rkiOf-0005ry-Ff
 for win-pv-devel@lists.xenproject.org; Thu, 14 Mar 2024 10:33:13 +0000
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
 [2a00:1450:4864:20::131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 42888299-e1ee-11ee-afdd-a90da7624cb6;
 Thu, 14 Mar 2024 11:33:13 +0100 (CET)
Received: by mail-lf1-x131.google.com with SMTP id
 2adb3069b0e04-513173e8191so1170013e87.1
 for <win-pv-devel@lists.xenproject.org>; Thu, 14 Mar 2024 03:33:13 -0700 (PDT)
Received: from [10.95.157.67] (54-240-197-239.amazon.com. [54.240.197.239])
 by smtp.gmail.com with ESMTPSA id
 o18-20020a05600c4fd200b004132f9cf053sm5155658wmq.33.2024.03.14.03.33.11
 for <win-pv-devel@lists.xenproject.org>
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 14 Mar 2024 03:33:12 -0700 (PDT)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 42888299-e1ee-11ee-afdd-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1710412392; x=1711017192; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:organization:reply-to
         :references:to:content-language:subject:user-agent:mime-version:date
         :message-id:from:from:to:cc:subject:date:message-id:reply-to;
        bh=COA//8kZyjp8CVcgiPItW+rBL8LZ3PKH8tEJiFHLbK8=;
        b=O2xXRq3nGq2BnZoe+0q39o21giifHz+kKLnmE325XWWfltWUaIyA7PHx2PxIeIMsOE
         n4Jo6cXj2a1C8zhfft27UZ6jmBYrDHKGQwmVpW7A+nek1ELdi/4glh7y2TRmBRcgs5+b
         DnyVnicAUS5JDc5hNbjsY1pr1xdOFkbnZWA9dj0hYXvWK2XkZbmia41EKcC2Dqt9F32u
         Fi+F2CVGbxe0/4Wg1jBafPyRi0xArg94UsLHng1JR2i/l3Lo845rpExNZIcTASUD8kAP
         oxcC05Z85ZV9MeHWAicarPSuEV/Ewzr8u9yYn7CnJutZOqIwuQv0GulzeFLjf+zJQEzK
         SLIg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1710412392; x=1711017192;
        h=content-transfer-encoding:in-reply-to:organization:reply-to
         :references:to:content-language:subject:user-agent:mime-version:date
         :message-id:from:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=COA//8kZyjp8CVcgiPItW+rBL8LZ3PKH8tEJiFHLbK8=;
        b=FgQhx4LChfP120LDClMqve1sZmSViJei8+oR0EY0NjxrooMlR8d5NYxfylkGJyLdDE
         UGTtNCnn7mOZiMc8BSJ7AaDvxWRj6QkGoIORGps49OdtvgWwCyaL4+oTaSaV1jIlnSmJ
         ZuSFdeYQq5xLLESZr4T61Ypb7Qv5zYAtROK3ho0xBlx1mCo+cGJDFCAjNbvcky9ThK1C
         pEP+hLGuSZzCwuRfcBqor8j+rX5fTRKKfs+Fvp8r7W3PtpAM550uGO/qP4jDDqP6D/v4
         oTRlGWvRUtJTnkWcXnsokHcF/bBOKnOobo8wWfInzZMMrZMPu4xqNCL+mIzkzBGq5TjO
         5L4A==
X-Gm-Message-State: AOJu0Yw2GZ5UcAftN4rnpuvF0Mw1vXamSpbmAfY4N2q6uvC0mEKB8Qbz
	4uyKB8FRJ6vk7PydecWeRl9LtWGyEVxSETatbBgir52jAieEPDveY8sK1b50Al8=
X-Google-Smtp-Source: AGHT+IG7wyfeqTlQPwtQb968l8gKl4MjfBVqVcamFp5sBYKYTpHNcfJ0zwZuahHbRAtTq3B2ayls6w==
X-Received: by 2002:a05:6512:557:b0:513:d339:8b1 with SMTP id h23-20020a056512055700b00513d33908b1mr578608lfl.27.1710412392418;
        Thu, 14 Mar 2024 03:33:12 -0700 (PDT)
From: Paul Durrant <xadimgnik@gmail.com>
X-Google-Original-From: Paul Durrant <paul@xen.org>
Message-ID: <dfc8160c-95c8-49df-bc21-abc4bd744a60@xen.org>
Date: Thu, 14 Mar 2024 10:33:06 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/2] Remove unnecessary assignment of EthernetHeader
Content-Language: en-US
To: win-pv-devel@lists.xenproject.org
References: <20240305153331.1555-1-owen.smith@cloud.com>
 <20240305153331.1555-2-owen.smith@cloud.com>
Reply-To: paul@xen.org
Organization: Xen Project
In-Reply-To: <20240305153331.1555-2-owen.smith@cloud.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 05/03/2024 15:33, Owen Smith wrote:
> BaseVa and EthernetHeader are not used after this point in the function,
> remove the unnecessary assignment
> 
> Signed-off-by: Owen Smith <owen.smith@cloud.com>

Reviewed-by: Paul Durrant <paul@xen.org>

> ---
>   src/xenvif/receiver.c | 4 ----
>   1 file changed, 4 deletions(-)
> 
> diff --git a/src/xenvif/receiver.c b/src/xenvif/receiver.c
> index 37e2c60..df5ed6f 100644
> --- a/src/xenvif/receiver.c
> +++ b/src/xenvif/receiver.c
> @@ -469,10 +469,6 @@ ReceiverRingProcessTag(
>   
>       Info->Length -= sizeof (ETHERNET_TAG);
>   
> -    BaseVa += Packet->Offset;
> -
> -    EthernetHeader = (PETHERNET_HEADER)(BaseVa + Info->EthernetHeader.Offset);
> -
>       ASSERT3U(PayloadLength, ==, Packet->Length - Info->Length);
>   }
>   



From win-pv-devel-bounces@lists.xenproject.org Fri Mar 15 10:12:07 2024
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Fri, 15 Mar 2024 10:12:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.693731.1082167 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rl4Xl-0000o6-H5; Fri, 15 Mar 2024 10:12:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 693731.1082167; Fri, 15 Mar 2024 10:12:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rl4Xl-0000nz-EU; Fri, 15 Mar 2024 10:12:05 +0000
Received: by outflank-mailman (input) for mailman id 693731;
 Fri, 15 Mar 2024 10:12:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Aglz=KV=invisiblethingslab.com=omeg@srs-se1.protection.inumbo.net>)
 id 1rl4Xj-0000nr-8c
 for win-pv-devel@lists.xenproject.org; Fri, 15 Mar 2024 10:12:03 +0000
Received: from wout1-smtp.messagingengine.com (wout1-smtp.messagingengine.com
 [64.147.123.24]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7547db97-e2b4-11ee-a1ee-f123f15fe8a2;
 Fri, 15 Mar 2024 11:12:00 +0100 (CET)
Received: from compute7.internal (compute7.nyi.internal [10.202.2.48])
 by mailout.west.internal (Postfix) with ESMTP id 2CC653200A01;
 Fri, 15 Mar 2024 06:11:57 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163])
 by compute7.internal (MEProxy); Fri, 15 Mar 2024 06:11:57 -0400
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri,
 15 Mar 2024 06:11:55 -0400 (EDT)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7547db97-e2b4-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:message-id:mime-version:reply-to:subject:subject:to:to; s=fm1;
	 t=1710497516; x=1710583916; bh=5mYPObzCoGMzXoPx1LDiDysM6PQl0Xr8
	gZDPL8aQHJg=; b=L0c02KcYHs+KAMEJEomjao5OjJZco4/Oshg4iylsmzB5a+cj
	ukak9wdbIhW+BiHwZVIuTgqKAFHQCpLy4SWKQiY+zmqIk6EQd3RL2JkE2kFSbiNZ
	BHzoT29vvVYPm/99fIgdOXPbBT/nayahQAZ06DUnb5wgjM5k+oiPgJcEg5ROEOrG
	yKeMZcr2Cf/BuGF+ai/HaEzkp3qF75+7MK4bYZFTPxO0jpK2Yh84iu602UR31yCg
	ZpLGPJrlSdULKN7Fo8+f4p8lIK3nv+1qXU7lbMFSl1Gmcg0tT+GD0Yt/bv2/GUY7
	g48mqIjmKHbsc0myrSIu8Rm9EfxaaZGXSkKqJw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:message-id:mime-version:reply-to:subject
	:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender
	:x-sasl-enc; s=fm1; t=1710497516; x=1710583916; bh=5mYPObzCoGMzX
	oPx1LDiDysM6PQl0Xr8gZDPL8aQHJg=; b=g7b0M4Zg3MhR5gHCbWtU5bDVzDsQA
	y0G2VCDrss0CAZa8/FvttLNJ6N/skJoRA7GckkPSSBaJOBTm1pzsxe1rv4YbPig1
	MSNyhzvC+oVN3kUzFDJenJiVIfcZ+PyRCMaWax37l6oNF94ys37y8lEjmtT15G8f
	SVU5FDvW0vN+0RpQ4an+8ccSma3jIJUKeneVx4lYzndZ8iLkSOuUYN5KgTEmd1P1
	2b9+3UE0CTT/WBjGoPaaVRh651LvpKZZnqECNbP0P1Bh3BUywrfOZ4cNx/IjEvlS
	nJnkOkFOILg/T07MH2XnP5DvcFl3R85vOTH+7vXFLmaGwrewXnui5nMDw==
X-ME-Sender: <xms:7B70ZZeEH4VR9XaH_0UmpH9LrtM3nNDwaXTNA_1_a57RYoaJcSLgWA>
    <xme:7B70ZXM6tXRu-8u-RksadI1OJn48zhHI8ViXbZuX9Ir6_VGI-dm1xksmKyOcMQWTb
    znARWTC2rz20FQ>
X-ME-Received: <xmr:7B70ZSiNEReaFWxF8NWzthyCkMKiw-V__kQner74WsIVwajxC2sREu3E-6rsZY3Mi1G4xE-FGNcNrz8VDtKLHpkb6aoxNh-f_7Aerl4EHO0>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrjeelgdduvdcutefuodetggdotefrodftvf
    curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu
    uegrihhlohhuthemuceftddtnecunecujfgurhephffvvefufffkofggtgfgsehtkeertd
    ertdejnecuhfhrohhmpeftrghfrghlucghohhjugihlhgruceoohhmvghgsehinhhvihhs
    ihgslhgvthhhihhnghhslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhepfffhkeegle
    fgvedtvdfggfejteffgfefgfegjeeulefhgfdvheeggfegfedtuedtnecuvehluhhsthgv
    rhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepohhmvghgsehinhhvihhsih
    gslhgvthhhihhnghhslhgrsgdrtghomh
X-ME-Proxy: <xmx:7B70ZS92Dq1qZv5KF8VQfmfJLZpNypT2WLjd-nXP-lDY95geBn46Xw>
    <xmx:7B70ZVtQ7p6ignFaUv1Bs-hk3JNqseQBQnXXPJu7BCPizh214SbuJA>
    <xmx:7B70ZRGVdEY5N1jPjLpNIzGtm-C5p6tMLXmpz8XMkGE1Vsji2ODTpg>
    <xmx:7B70ZcNP2lmbFk6U1mMFzVEV58VCMQES5du8d0e6up86OcwtY_4kHA>
    <xmx:7B70ZT0nBEewap7mI3LZnY3HDUF9nAv3AaytY767VEH3R0-ZXSsd-g>
Feedback-ID: i409c4082:Fastmail
From: =?UTF-8?q?Rafa=C5=82=20Wojdy=C5=82a?= <omeg@invisiblethingslab.com>
To: win-pv-devel@lists.xenproject.org
Cc: =?UTF-8?q?Rafa=C5=82=20Wojdy=C5=82a?= <omeg@invisiblethingslab.com>
Subject: [PATCH] xencontrol: Fix return values and simplify failure paths
Date: Fri, 15 Mar 2024 11:11:33 +0100
Message-ID: <20240315101133.1286-1-omeg@invisiblethingslab.com>
X-Mailer: git-send-email 2.43.0.windows.1
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

XcGnttabRevokeForeignAccess() and XcGnttabUnmapForeignPages()
could return error values even when the calls succeeded. This was because
GetLastError() was used to get the return value even if the previous call
to DeviceIoControl() succeeded.

This commit also reworks all xencontrol functions to use simpler control
paths in failure cases.

Signed-off-by: Rafał Wojdyła <omeg@invisiblethingslab.com>
---
 src/xencontrol/xencontrol.c | 268 ++++++++++++++++--------------------
 1 file changed, 116 insertions(+), 152 deletions(-)

diff --git a/src/xencontrol/xencontrol.c b/src/xencontrol/xencontrol.c
index fe39e87..c8a954c 100644
--- a/src/xencontrol/xencontrol.c
+++ b/src/xencontrol/xencontrol.c
@@ -94,49 +94,54 @@ XcOpen(
     SP_DEVICE_INTERFACE_DETAIL_DATA *DetailData = NULL;
     DWORD BufferSize;
     PXENCONTROL_CONTEXT Context;
+    DWORD Status = ERROR_OUTOFMEMORY;
 
     Context = malloc(sizeof(*Context));
     if (Context == NULL)
-        return ERROR_NOT_ENOUGH_MEMORY;
+        goto end;
 
     Context->Logger = Logger;
     Context->LogLevel = XLL_INFO;
 
     DevInfo = SetupDiGetClassDevs(&GUID_INTERFACE_XENIFACE, 0, NULL, DIGCF_PRESENT | DIGCF_DEVICEINTERFACE);
     if (DevInfo == INVALID_HANDLE_VALUE) {
+        Status = GetLastError();
         _Log(Logger, XLL_ERROR, Context->LogLevel, __FUNCTION__,
-             L"XENIFACE device class doesn't exist");
-        goto fail;
+             L"XENIFACE device class doesn't exist: 0x%x", Status);
+        goto end;
     }
 
     InterfaceData.cbSize = sizeof(InterfaceData);
     if (!SetupDiEnumDeviceInterfaces(DevInfo, NULL, &GUID_INTERFACE_XENIFACE, 0, &InterfaceData)) {
+        Status = GetLastError();
         _Log(Logger, XLL_ERROR, Context->LogLevel, __FUNCTION__,
-             L"Failed to enumerate XENIFACE devices");
-        goto fail;
+             L"Failed to enumerate XENIFACE devices: 0x%x", Status);
+        goto end;
     }
 
     SetupDiGetDeviceInterfaceDetail(DevInfo, &InterfaceData, NULL, 0, &BufferSize, NULL);
-    if (GetLastError() != ERROR_INSUFFICIENT_BUFFER) {
+    Status = GetLastError();
+    if (Status != ERROR_INSUFFICIENT_BUFFER) {
         _Log(Logger, XLL_ERROR, Context->LogLevel, __FUNCTION__,
-             L"Failed to get buffer size for device details");
-        goto fail;
+             L"Failed to get buffer size for device details: 0x%x", Status);
+        goto end;
     }
 
     // Using 'BufferSize' from failed function call
 #pragma warning(suppress: 6102)
     DetailData = (SP_DEVICE_INTERFACE_DETAIL_DATA *)malloc(BufferSize);
     if (!DetailData) {
-        SetLastError(ERROR_OUTOFMEMORY);
-        goto fail;
+        Status = ERROR_OUTOFMEMORY;
+        goto end;
     }
 
     DetailData->cbSize = sizeof(SP_DEVICE_INTERFACE_DETAIL_DATA);
 
     if (!SetupDiGetDeviceInterfaceDetail(DevInfo, &InterfaceData, DetailData, BufferSize, NULL, NULL)) {
+        Status = GetLastError();
         _Log(Logger, XLL_ERROR, Context->LogLevel, __FUNCTION__,
-             L"Failed to get XENIFACE device path");
-        goto fail;
+             L"Failed to get XENIFACE device path: 0x%x", Status);
+        goto end;
     }
 
     Context->XenIface = CreateFile(DetailData->DevicePath,
@@ -148,26 +153,27 @@ XcOpen(
                                    NULL);
 
     if (Context->XenIface == INVALID_HANDLE_VALUE) {
+        Status = GetLastError();
         _Log(Logger, XLL_ERROR, Context->LogLevel, __FUNCTION__,
-             L"Failed to open XENIFACE device, path: %s", DetailData->DevicePath);
-        goto fail;
+             L"Failed to open XENIFACE device (path: %s): 0x%x", DetailData->DevicePath, Status);
+        goto end;
     }
 
-    _Log(Logger, XLL_ERROR, Context->LogLevel, __FUNCTION__,
+    _Log(Logger, XLL_INFO, Context->LogLevel, __FUNCTION__,
          L"XenIface handle: %p", Context->XenIface);
 
-    free(DetailData);
     *Xc = Context;
-    return ERROR_SUCCESS;
-
-fail:
-    _Log(Logger, XLL_ERROR, Context->LogLevel, __FUNCTION__,
-         L"Error: 0x%x", GetLastError());
+     Status = ERROR_SUCCESS;
 
+end:
     free(DetailData);
-    free(Context);
-    *Xc = NULL;
-    return GetLastError();
+
+    if (Status != ERROR_SUCCESS) {
+        free(Context);
+        *Xc = NULL;
+    }
+
+    return Status;
 }
 
 void
@@ -192,6 +198,7 @@ XcEvtchnOpenUnbound(
     XENIFACE_EVTCHN_BIND_UNBOUND_OUT Out;
     DWORD Returned;
     BOOL Success;
+    DWORD Status = ERROR_SUCCESS;
 
     In.RemoteDomain = RemoteDomain;
     In.Event = Event;
@@ -206,18 +213,16 @@ XcEvtchnOpenUnbound(
                               NULL);
 
     if (!Success) {
-        Log(XLL_ERROR, L"IOCTL_XENIFACE_EVTCHN_BIND_UNBOUND failed");
-        goto fail;
+        Status = GetLastError();
+        Log(XLL_ERROR, L"IOCTL_XENIFACE_EVTCHN_BIND_UNBOUND failed: 0x%x", Status);
+        goto end;
     }
 
     *LocalPort = Out.LocalPort;
     Log(XLL_DEBUG, L"LocalPort: %lu", *LocalPort);
 
-    return ERROR_SUCCESS;
-
-fail:
-    Log(XLL_ERROR, L"Error: 0x%x", GetLastError());
-    return GetLastError();
+end:
+    return Status;
 }
 
 DWORD
@@ -234,6 +239,7 @@ XcEvtchnBindInterdomain(
     XENIFACE_EVTCHN_BIND_INTERDOMAIN_OUT Out;
     DWORD Returned;
     BOOL Success;
+    DWORD Status = ERROR_SUCCESS;
 
     In.RemoteDomain = RemoteDomain;
     In.RemotePort = RemotePort;
@@ -250,18 +256,16 @@ XcEvtchnBindInterdomain(
                               NULL);
 
     if (!Success) {
-        Log(XLL_ERROR, L"IOCTL_XENIFACE_EVTCHN_BIND_INTERDOMAIN failed");
-        goto fail;
+        Status = GetLastError();
+        Log(XLL_ERROR, L"IOCTL_XENIFACE_EVTCHN_BIND_INTERDOMAIN failed: 0x%x", Status);
+        goto end;
     }
 
     *LocalPort = Out.LocalPort;
     Log(XLL_DEBUG, L"LocalPort: %lu", *LocalPort);
 
-    return ERROR_SUCCESS;
-
-fail:
-    Log(XLL_ERROR, L"Error: 0x%x", GetLastError());
-    return GetLastError();
+end:
+    return Status;
 }
 
 DWORD
@@ -273,6 +277,7 @@ XcEvtchnClose(
     XENIFACE_EVTCHN_CLOSE_IN In;
     DWORD Returned;
     BOOL Success;
+    DWORD Status = ERROR_SUCCESS;
 
     In.LocalPort = LocalPort;
 
@@ -285,15 +290,11 @@ XcEvtchnClose(
                               NULL);
 
     if (!Success) {
-        Log(XLL_ERROR, L"IOCTL_XENIFACE_EVTCHN_CLOSE failed");
-        goto fail;
+        Status = GetLastError();
+        Log(XLL_ERROR, L"IOCTL_XENIFACE_EVTCHN_CLOSE failed: 0x%x", Status);
     }
 
-    return ERROR_SUCCESS;
-
-fail:
-    Log(XLL_ERROR, L"Error: 0x%x", GetLastError());
-    return GetLastError();
+    return Status;
 }
 
 DWORD
@@ -305,6 +306,7 @@ XcEvtchnNotify(
     XENIFACE_EVTCHN_NOTIFY_IN In;
     DWORD Returned;
     BOOL Success;
+    DWORD Status = ERROR_SUCCESS;
 
     In.LocalPort = LocalPort;
 
@@ -317,15 +319,11 @@ XcEvtchnNotify(
                               NULL);
 
     if (!Success) {
-        Log(XLL_ERROR, L"IOCTL_XENIFACE_EVTCHN_NOTIFY failed");
-        goto fail;
+        Status = GetLastError();
+        Log(XLL_ERROR, L"IOCTL_XENIFACE_EVTCHN_NOTIFY failed: 0x%x", Status);
     }
 
-    return ERROR_SUCCESS;
-
-fail:
-    Log(XLL_ERROR, L"Error: 0x%x", GetLastError());
-    return GetLastError();
+    return Status;
 }
 
 DWORD
@@ -337,6 +335,7 @@ XcEvtchnUnmask(
     XENIFACE_EVTCHN_UNMASK_IN In;
     DWORD Returned;
     BOOL Success;
+    DWORD Status = ERROR_SUCCESS;
 
     In.LocalPort = LocalPort;
 
@@ -349,15 +348,11 @@ XcEvtchnUnmask(
                               NULL);
 
     if (!Success) {
-        Log(XLL_ERROR, L"IOCTL_XENIFACE_EVTCHN_UNMASK failed");
-        goto fail;
+        Status = GetLastError();
+        Log(XLL_ERROR, L"IOCTL_XENIFACE_EVTCHN_UNMASK failed: 0x%x", Status);
     }
 
-    return ERROR_SUCCESS;
-
-fail:
-    Log(XLL_ERROR, L"Error: 0x%x", GetLastError());
-    return GetLastError();
+    return Status;
 }
 
 DWORD
@@ -418,7 +413,7 @@ XcGnttabPermitForeignAccess2(
 
     Status = ERROR_OUTOFMEMORY;
     if (!Out)
-        goto fail;
+        goto end;
 
     Log(XLL_DEBUG, L"RemoteDomain: %d, Address %p, NumberPages: %lu, NotifyOffset: 0x%x, NotifyPort: %lu, Flags: 0x%x",
         RemoteDomain, Address, NumberPages, NotifyOffset, NotifyPort, Flags);
@@ -431,17 +426,18 @@ XcGnttabPermitForeignAccess2(
                               &Returned,
                               &Overlapped);
 
-    Status = GetLastError();
     // this IOCTL is expected to be pending on success
     if (!Success) {
+        Status = GetLastError();
         if (Status != ERROR_IO_PENDING) {
-            Log(XLL_ERROR, L"IOCTL_XENIFACE_GNTTAB_PERMIT_FOREIGN_ACCESS_V2 failed");
-            goto fail;
+            Log(XLL_ERROR, L"IOCTL_XENIFACE_GNTTAB_PERMIT_FOREIGN_ACCESS_V2 failed: 0x%x", Status);
+            goto end;
         }
+        Status = ERROR_SUCCESS;
     } else {
         Log(XLL_ERROR, L"IOCTL_XENIFACE_GNTTAB_PERMIT_FOREIGN_ACCESS_V2 not pending");
         Status = ERROR_UNIDENTIFIED_ERROR;
-        goto fail;
+        goto end;
     }
 
     *SharedAddress = Out->Address;
@@ -452,11 +448,7 @@ XcGnttabPermitForeignAccess2(
         Log(XLL_DEBUG, L"Grant ref[%lu]: %lu", i, Out->References[i]);
 #endif
 
-    free(Out);
-    return ERROR_SUCCESS;
-
-fail:
-    Log(XLL_ERROR, L"Error: 0x%x", Status);
+end:
     free(Out);
     return Status;
 }
@@ -470,7 +462,7 @@ XcGnttabRevokeForeignAccess(
     XENIFACE_GNTTAB_REVOKE_FOREIGN_ACCESS_IN_V2 In;
     DWORD Returned;
     BOOL Success;
-    DWORD Status;
+    DWORD Status = ERROR_SUCCESS;
 
     Log(XLL_DEBUG, L"Address: %p", Address);
     In.Address = Address;
@@ -482,17 +474,12 @@ XcGnttabRevokeForeignAccess(
                               &Returned,
                               NULL);
 
-    Status = GetLastError();
     if (!Success) {
-        Log(XLL_ERROR, L"IOCTL_XENIFACE_GNTTAB_REVOKE_FOREIGN_ACCESS_V2 failed");
-        goto fail;
+        Status = GetLastError();
+        Log(XLL_ERROR, L"IOCTL_XENIFACE_GNTTAB_REVOKE_FOREIGN_ACCESS_V2 failed: 0x%x", Status);
     }
 
     return Status;
-
-fail:
-    Log(XLL_ERROR, L"Error: %d 0x%x", Status, Status);
-    return Status;
 }
 
 DWORD
@@ -512,13 +499,12 @@ XcGnttabMapForeignPages(
     DWORD Returned, Size;
     OVERLAPPED Overlapped;
     BOOL Success;
-    DWORD Status;
+    DWORD Status = ERROR_OUTOFMEMORY;
 
-    Status = ERROR_OUTOFMEMORY;
     Size = (ULONG)FIELD_OFFSET(XENIFACE_GNTTAB_MAP_FOREIGN_PAGES_IN_V2, References[NumberPages]);
     In = malloc(Size);
     if (!In)
-        goto fail;
+        goto end;
 
     In->RemoteDomain = RemoteDomain;
     In->NumberPages = NumberPages;
@@ -543,28 +529,25 @@ XcGnttabMapForeignPages(
                               &Returned,
                               &Overlapped);
 
-    Status = GetLastError();
     // this IOCTL is expected to be pending on success
     if (!Success) {
+        Status = GetLastError();
         if (Status != ERROR_IO_PENDING) {
-            Log(XLL_ERROR, L"IOCTL_XENIFACE_GNTTAB_MAP_FOREIGN_PAGES_V2 failed");
-            goto fail;
+            Log(XLL_ERROR, L"IOCTL_XENIFACE_GNTTAB_MAP_FOREIGN_PAGES_V2 failed: 0x%x", Status);
+            goto end;
         }
+        Status = ERROR_SUCCESS;
     } else {
         Log(XLL_ERROR, L"IOCTL_XENIFACE_GNTTAB_MAP_FOREIGN_PAGES_V2 not pending");
         Status = ERROR_UNIDENTIFIED_ERROR;
-        goto fail;
+        goto end;
     }
 
     *Address = Out.Address;
 
     Log(XLL_DEBUG, L"Address: %p", *Address);
 
-    free(In);
-    return ERROR_SUCCESS;
-
-fail:
-    Log(XLL_ERROR, L"Error: 0x%x", Status);
+end:
     free(In);
     return Status;
 }
@@ -578,7 +561,7 @@ XcGnttabUnmapForeignPages(
     XENIFACE_GNTTAB_UNMAP_FOREIGN_PAGES_IN_V2 In;
     DWORD Returned;
     BOOL Success;
-    DWORD Status;
+    DWORD Status = ERROR_SUCCESS;
 
     Log(XLL_DEBUG, L"Address: %p", Address);
 
@@ -591,17 +574,12 @@ XcGnttabUnmapForeignPages(
                               &Returned,
                               NULL);
 
-    Status = GetLastError();
     if (!Success) {
-        Log(XLL_ERROR, L"IOCTL_XENIFACE_GNTTAB_UNMAP_FOREIGN_PAGES_V2 failed");
-        goto fail;
+        Status = GetLastError();
+        Log(XLL_ERROR, L"IOCTL_XENIFACE_GNTTAB_UNMAP_FOREIGN_PAGES_V2 failed: 0x%x", Status);
     }
 
     return Status;
-
-fail:
-    Log(XLL_ERROR, L"Error: 0x%x", Status);
-    return Status;
 }
 
 DWORD
@@ -614,6 +592,7 @@ XcStoreRead(
 {
     DWORD Returned;
     BOOL Success;
+    DWORD Status = ERROR_SUCCESS;
 
     Log(XLL_DEBUG, L"Path: '%S'", Path);
     Success = DeviceIoControl(Xc->XenIface,
@@ -624,17 +603,15 @@ XcStoreRead(
                               NULL);
 
     if (!Success) {
-        Log(XLL_ERROR, L"IOCTL_XENIFACE_STORE_READ failed");
-        goto fail;
+        Status = GetLastError();
+        Log(XLL_ERROR, L"IOCTL_XENIFACE_STORE_READ failed: 0x%x", Status);
+        goto end;
     }
 
     Log(XLL_DEBUG, L"Value: '%S'", Value);
 
-    return ERROR_SUCCESS;
-
-fail:
-    Log(XLL_ERROR, L"Error: 0x%x", GetLastError());
-    return GetLastError();
+end:
+    return Status;
 }
 
 DWORD
@@ -648,12 +625,13 @@ XcStoreWrite(
     DWORD cbBuffer;
     DWORD Returned;
     BOOL Success;
+    DWORD Status = ERROR_SUCCESS;
 
     cbBuffer = (DWORD)(strlen(Path) + 1 + strlen(Value) + 1 + 1);
     Buffer = malloc(cbBuffer);
     if (!Buffer) {
-        SetLastError(ERROR_OUTOFMEMORY);
-        goto fail;
+        Status = ERROR_OUTOFMEMORY;
+        goto end;
     }
 
     ZeroMemory(Buffer, cbBuffer);
@@ -669,17 +647,14 @@ XcStoreWrite(
                               NULL);
 
     if (!Success) {
-        Log(XLL_ERROR, L"IOCTL_XENIFACE_STORE_WRITE failed");
-        goto fail;
+        Status = GetLastError();
+        Log(XLL_ERROR, L"IOCTL_XENIFACE_STORE_WRITE failed: 0x%x", Status);
     }
 
     free(Buffer);
-    return ERROR_SUCCESS;
 
-fail:
-    Log(XLL_ERROR, L"Error: 0x%x", GetLastError());
-    free(Buffer);
-    return GetLastError();
+end:
+    return Status;
 }
 
 DWORD
@@ -692,6 +667,7 @@ XcStoreDirectory(
 {
     DWORD Returned;
     BOOL Success;
+    DWORD Status = ERROR_SUCCESS;
 
     Log(XLL_DEBUG, L"Path: '%S'", Path);
     Success = DeviceIoControl(Xc->XenIface,
@@ -702,17 +678,15 @@ XcStoreDirectory(
                               NULL);
 
     if (!Success) {
-        Log(XLL_ERROR, L"IOCTL_XENIFACE_STORE_DIRECTORY failed");
-        goto fail;
+        Status = GetLastError();
+        Log(XLL_ERROR, L"IOCTL_XENIFACE_STORE_DIRECTORY failed: 0x%x", Status);
+        goto end;
     }
 
     _LogMultiSz(Xc, __FUNCTION__, XLL_DEBUG, Output);
 
-    return ERROR_SUCCESS;
-
-fail:
-    Log(XLL_ERROR, L"Error: 0x%x", GetLastError());
-    return GetLastError();
+end:
+    return Status;
 }
 
 DWORD
@@ -723,6 +697,7 @@ XcStoreRemove(
 {
     DWORD Returned;
     BOOL Success;
+    DWORD Status = ERROR_SUCCESS;
 
     Log(XLL_DEBUG, L"Path: '%S'", Path);
     Success = DeviceIoControl(Xc->XenIface,
@@ -733,15 +708,11 @@ XcStoreRemove(
                               NULL);
 
     if (!Success) {
-        Log(XLL_ERROR, L"IOCTL_XENIFACE_STORE_REMOVE failed");
-        goto fail;
+        Status = GetLastError();
+        Log(XLL_ERROR, L"IOCTL_XENIFACE_STORE_REMOVE failed: 0x%x", Status);
     }
 
-    return ERROR_SUCCESS;
-
-fail:
-    Log(XLL_ERROR, L"Error: 0x%x", GetLastError());
-    return GetLastError();
+    return Status;
 }
 
 DWORD
@@ -754,6 +725,7 @@ XcStoreSetPermissions(
 {
     DWORD Returned, Size;
     BOOL Success;
+    DWORD Status = ERROR_SUCCESS;
     XENIFACE_STORE_SET_PERMISSIONS_IN *In = NULL;
 
     Log(XLL_DEBUG, L"Path: '%S', Count: %lu", Path, Count);
@@ -763,8 +735,8 @@ XcStoreSetPermissions(
     Size = (ULONG)FIELD_OFFSET(XENIFACE_STORE_SET_PERMISSIONS_IN, Permissions[Count]);
     In = malloc(Size);
     if (!In) {
-        SetLastError(ERROR_OUTOFMEMORY);
-        goto fail;
+        Status = ERROR_OUTOFMEMORY;
+        goto end;
     }
 
     In->Path = Path;
@@ -780,17 +752,13 @@ XcStoreSetPermissions(
                               NULL);
 
     if (!Success) {
-        Log(XLL_ERROR, L"IOCTL_XENIFACE_STORE_SET_PERMISSIONS failed");
-        goto fail;
+        Status = GetLastError();
+        Log(XLL_ERROR, L"IOCTL_XENIFACE_STORE_SET_PERMISSIONS failed: 0x%x", Status);
     }
 
     free(In);
-    return ERROR_SUCCESS;
-
-fail:
-    Log(XLL_ERROR, L"Error: 0x%x", GetLastError());
-    free(In);
-    return GetLastError();
+end:
+    return Status;
 }
 
 DWORD
@@ -803,6 +771,7 @@ XcStoreAddWatch(
 {
     DWORD Returned;
     BOOL Success;
+    DWORD Status = ERROR_SUCCESS;
     XENIFACE_STORE_ADD_WATCH_IN In;
     XENIFACE_STORE_ADD_WATCH_OUT Out;
 
@@ -819,19 +788,17 @@ XcStoreAddWatch(
                               NULL);
 
     if (!Success) {
-        Log(XLL_ERROR, L"IOCTL_XENIFACE_STORE_ADD_WATCH failed");
-        goto fail;
+        Status = GetLastError();
+        Log(XLL_ERROR, L"IOCTL_XENIFACE_STORE_ADD_WATCH failed: 0x%x", Status);
+        goto end;
     }
 
     *Handle = Out.Context;
 
-    Log(XLL_DEBUG, L"Handle: %p", *Handle);
-
-    return ERROR_SUCCESS;
+    Log(XLL_DEBUG, L"Watch handle: %p", *Handle);
 
-fail:
-    Log(XLL_ERROR, L"Error: 0x%x", GetLastError());
-    return GetLastError();
+end:
+    return Status;
 }
 
 DWORD
@@ -842,6 +809,7 @@ XcStoreRemoveWatch(
 {
     DWORD Returned;
     BOOL Success;
+    DWORD Status = ERROR_SUCCESS;
     XENIFACE_STORE_REMOVE_WATCH_IN In;
 
     Log(XLL_DEBUG, L"Handle: %p", Handle);
@@ -855,13 +823,9 @@ XcStoreRemoveWatch(
                               NULL);
 
     if (!Success) {
-        Log(XLL_ERROR, L"IOCTL_XENIFACE_STORE_REMOVE_WATCH failed");
-        goto fail;
+        Status = GetLastError();
+        Log(XLL_ERROR, L"IOCTL_XENIFACE_STORE_REMOVE_WATCH failed: 0x%x", Status);
     }
 
-    return ERROR_SUCCESS;
-
-fail:
-    Log(XLL_ERROR, L"Error: 0x%x", GetLastError());
-    return GetLastError();
+    return Status;
 }
-- 
2.43.0.windows.1



From win-pv-devel-bounces@lists.xenproject.org Mon Mar 18 15:29:11 2024
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Mon, 18 Mar 2024 15:29:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.694790.1083923 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rmEvF-0007Mh-3C; Mon, 18 Mar 2024 15:29:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 694790.1083923; Mon, 18 Mar 2024 15:29:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rmEvF-0007Ma-0N; Mon, 18 Mar 2024 15:29:09 +0000
Received: by outflank-mailman (input) for mailman id 694790;
 Mon, 18 Mar 2024 15:29:08 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CJaA=KY=cloud.com=owen.smith@srs-se1.protection.inumbo.net>)
 id 1rmEvD-0007MU-QW
 for win-pv-devel@lists.xenproject.org; Mon, 18 Mar 2024 15:29:08 +0000
Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com
 [2a00:1450:4864:20::32e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 408ec714-e53c-11ee-afdd-a90da7624cb6;
 Mon, 18 Mar 2024 16:29:05 +0100 (CET)
Received: by mail-wm1-x32e.google.com with SMTP id
 5b1f17b1804b1-41409fd8b6eso18954475e9.2
 for <win-pv-devel@lists.xenproject.org>; Mon, 18 Mar 2024 08:29:04 -0700 (PDT)
Received: from localhost.localdomain
 (host86-169-131-93.range86-169.btcentralplus.com. [86.169.131.93])
 by smtp.gmail.com with ESMTPSA id
 n18-20020a05600c4f9200b00414618bea7csm244053wmq.37.2024.03.18.08.29.02
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 18 Mar 2024 08:29:02 -0700 (PDT)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 408ec714-e53c-11ee-afdd-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1710775743; x=1711380543; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=+o7BIAhgwcb51+TfNoWUc1v17ESddRuyR3w1jHvm/4w=;
        b=EXQjH3a+5GY+TX/UZA+Xgw2Ei2vBfV7LTnl36bpLEOWWy+utr+SD9BGdyxXeVX/7j+
         875UyrBdxorpyvNiL/mOmQpo/u8+9E05/CwATKC4rjMMwI5SchdSma897ISw/7FxGx2a
         qtaItDdWU3y122wX3Y/nvbbGBPUj1fY0QwjeQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1710775743; x=1711380543;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=+o7BIAhgwcb51+TfNoWUc1v17ESddRuyR3w1jHvm/4w=;
        b=adxGU3hPt+obsZ99r95Qu0W3qj0oWcP641Cr3rrbE7OF98HxBqJBbG71soKfcEKVAN
         cWdWdugTO71jdK/Af1yYRG4CX75DnksgFz+ghRVhxNAZjhspkv990DA7vN6CPbDZ8DUk
         TOspnBU3+akUDb1DOdkIOWu1mOKMXgV14E15YI2qvT6YFKDKMYpl7xURa/tXi7CZ7R8A
         Bfp9WLyR1fap9+W8Y0HMNstXDKCqCp5EyHw/mD0ahQxeUq5Ml/mt/fvc7mWWPWTTiMBZ
         a2l3e+OSm0+SgKX3JqFckwREeDeA0gOtMcWwvIrYeuKgkpxOb5I3JiwNIWehKKUIJ0do
         tGkA==
X-Gm-Message-State: AOJu0YwI9tpQ8cAmNIkagxxnS7UwBvROC6NK94So5qX4YR9m2LL82AwW
	hHrCs/IFKmKbVnf0pSgfXl8nVRja5/ffu9LFbt0k6xLVFMn5zRxLGnLnSnVHGudTWv1SkVH7s0c
	5Jg==
X-Google-Smtp-Source: AGHT+IH/OGme/41YdX7XCjuX1SzlHqTd569ZH8n9y6mHc9iPiRsG+km+vL8hzrTCOGNY1rRxLkkWhA==
X-Received: by 2002:a05:600c:2545:b0:414:14:3b75 with SMTP id e5-20020a05600c254500b0041400143b75mr6958738wma.15.1710775742961;
        Mon, 18 Mar 2024 08:29:02 -0700 (PDT)
From: Owen Smith <owen.smith@cloud.com>
To: win-pv-devel@lists.xenproject.org
Cc: Martin Harvey <Martin.Harvey@citrix.com>,
	Martin Harvey <martin.harvey@citrix.com>,
	Owen Smith <owen.smith@cloud.com>
Subject: [PATCH] Asynchronous power handling.
Date: Mon, 18 Mar 2024 15:28:51 +0000
Message-ID: <20240318152851.1558-1-owen.smith@cloud.com>
X-Mailer: git-send-email 2.41.0.windows.3
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Martin Harvey <Martin.Harvey@citrix.com>

Use completion routines and IO_WORKITEMs to handle power state transitions,
allowing execution to run at PASSIVE_LEVEL without blocking the current thread,
and avoid blocking all power IRPs while another is executing (power IRPs should
be synchronized by the power manager, but this doesnt appear to be the case in
all situations).

Signed-off-by: Martin Harvey <martin.harvey@citrix.com>

Refactored, wrong format comments removed
Signed-off-by: Owen Smith <owen.smith@cloud.com>

---
 src/xenvif/fdo.c | 810 +++++++++++++++++++++++++++++------------------
 src/xenvif/pdo.c | 263 +++++++--------
 2 files changed, 602 insertions(+), 471 deletions(-)

diff --git a/src/xenvif/fdo.c b/src/xenvif/fdo.c
index 451d582..efcd131 100644
--- a/src/xenvif/fdo.c
+++ b/src/xenvif/fdo.c
@@ -81,9 +81,9 @@ struct _XENVIF_FDO {
     ULONG                       Usage[DeviceUsageTypeDumpFile + 1];
     BOOLEAN                     NotDisableable;
 
-    PXENVIF_THREAD              SystemPowerThread;
+    PIO_WORKITEM                SystemPowerWorkItem;
     PIRP                        SystemPowerIrp;
-    PXENVIF_THREAD              DevicePowerThread;
+    PIO_WORKITEM                DevicePowerWorkItem;
     PIRP                        DevicePowerIrp;
 
     CHAR                        VendorName[MAXNAMELEN];
@@ -2168,42 +2168,112 @@ FdoDispatchPnp(
     return status;
 }
 
-static FORCEINLINE NTSTATUS
-__FdoSetDevicePowerUp(
-    IN  PXENVIF_FDO     Fdo,
-    IN  PIRP            Irp
+__drv_functionClass(IO_WORKITEM_ROUTINE)
+__drv_sameIRQL
+static VOID
+FdoDevicePowerUpWorker(
+    IN  PDEVICE_OBJECT  DeviceObject,
+    IN  PVOID           Context
     )
 {
+    PXENVIF_FDO         Fdo = (PXENVIF_FDO) Context;
+    PIRP                Irp;
+
+    UNREFERENCED_PARAMETER(DeviceObject);
+
+    Irp = InterlockedExchangePointer(&Fdo->DevicePowerIrp, NULL);
+    ASSERT(Irp != NULL);
+
+    (VOID) FdoD3ToD0(Fdo);
+
+    IoCompleteRequest(Irp, IO_NO_INCREMENT);
+}
+
+__drv_functionClass(IO_COMPLETION_ROUTINE)
+__drv_sameIRQL
+static NTSTATUS
+FdoSetDevicePowerUpComplete(
+    IN  PDEVICE_OBJECT  DeviceObject,
+    IN  PIRP            Irp,
+    IN  PVOID           Context
+    )
+{
+    PXENVIF_FDO         Fdo = (PXENVIF_FDO) Context;
     PIO_STACK_LOCATION  StackLocation;
     DEVICE_POWER_STATE  DeviceState;
-    NTSTATUS            status;
+    PVOID               Exchange;
 
-    Trace("====>\n");
+    UNREFERENCED_PARAMETER(DeviceObject);
 
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
     DeviceState = StackLocation->Parameters.Power.State.DeviceState;
 
-    ASSERT3U(DeviceState, <,  __FdoGetDevicePowerState(Fdo));
-
-    status = FdoForwardIrpSynchronously(Fdo, Irp);
-    if (!NT_SUCCESS(status))
-        goto done;
-
     Info("%s: %s -> %s\n",
          __FdoGetName(Fdo),
          PowerDeviceStateName(__FdoGetDevicePowerState(Fdo)),
          PowerDeviceStateName(DeviceState));
 
     ASSERT3U(DeviceState, ==, PowerDeviceD0);
-    status = FdoD3ToD0(Fdo);
-    ASSERT(NT_SUCCESS(status));
 
-done:
-    Irp->IoStatus.Status = status;
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
+    Exchange = InterlockedExchangePointer(&Fdo->DevicePowerIrp, Irp);
+    ASSERT(Exchange == NULL);
 
-    Trace("<==== (%08x)\n", status);
-    return status;
+    IoQueueWorkItem(Fdo->DevicePowerWorkItem,
+                    FdoDevicePowerUpWorker,
+                    DelayedWorkQueue,
+                    Fdo);
+
+    return STATUS_MORE_PROCESSING_REQUIRED;
+}
+
+static FORCEINLINE NTSTATUS
+__FdoSetDevicePowerUp(
+    IN  PXENVIF_FDO     Fdo,
+    IN  PIRP            Irp
+    )
+{
+    PIO_STACK_LOCATION  StackLocation;
+    DEVICE_POWER_STATE  DeviceState;
+
+    StackLocation = IoGetCurrentIrpStackLocation(Irp);
+    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
+
+    ASSERT3U(DeviceState, <, __FdoGetDevicePowerState(Fdo));
+
+    IoMarkIrpPending(Irp);
+    IoCopyCurrentIrpStackLocationToNext(Irp);
+    IoSetCompletionRoutine(Irp,
+                           FdoSetDevicePowerUpComplete,
+                           Fdo,
+                           TRUE,
+                           TRUE,
+                           TRUE);
+
+    (VOID) IoCallDriver(Fdo->LowerDeviceObject, Irp);
+
+    return STATUS_PENDING;
+}
+
+__drv_functionClass(IO_WORKITEM_ROUTINE)
+__drv_sameIRQL
+static VOID
+FdoDevicePowerDownWorker(
+    IN  PDEVICE_OBJECT  DeviceObject,
+    IN  PVOID           Context
+    )
+{
+    PXENVIF_FDO         Fdo = (PXENVIF_FDO) Context;
+    PIRP                Irp;
+
+    UNREFERENCED_PARAMETER(DeviceObject);
+
+    Irp = InterlockedExchangePointer(&Fdo->DevicePowerIrp, NULL);
+    ASSERT(Irp != NULL);
+
+    FdoD0ToD3(Fdo);
+
+    IoCopyCurrentIrpStackLocationToNext(Irp);
+    IoCallDriver(Fdo->LowerDeviceObject, Irp);
 }
 
 static FORCEINLINE NTSTATUS
@@ -2228,11 +2298,24 @@ __FdoSetDevicePowerDown(
 
     ASSERT3U(DeviceState, ==, PowerDeviceD3);
 
-    if (__FdoGetDevicePowerState(Fdo) == PowerDeviceD0)
-        FdoD0ToD3(Fdo);
+    if (__FdoGetDevicePowerState(Fdo) != PowerDeviceD0) {
+        IoSkipCurrentIrpStackLocation(Irp);
+        status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
+    } else {
+        PVOID           Exchange;
 
-    IoSkipCurrentIrpStackLocation(Irp);
-    status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
+        IoMarkIrpPending(Irp);
+
+        Exchange = InterlockedExchangePointer(&Fdo->DevicePowerIrp, Irp);
+        ASSERT(Exchange == NULL);
+
+        IoQueueWorkItem(Fdo->DevicePowerWorkItem,
+                        FdoDevicePowerDownWorker,
+                        DelayedWorkQueue,
+                        Fdo);
+
+        status = STATUS_PENDING;
+    }
 
     return status;
 }
@@ -2252,12 +2335,11 @@ __FdoSetDevicePower(
     DeviceState = StackLocation->Parameters.Power.State.DeviceState;
     PowerAction = StackLocation->Parameters.Power.ShutdownType;
 
-    Trace("====> (%s:%s)\n",
-          PowerDeviceStateName(DeviceState), 
+    Trace("%s: ====> (%s:%s)\n",
+          __FdoGetName(Fdo),
+          PowerDeviceStateName(DeviceState),
           PowerActionName(PowerAction));
 
-    ASSERT3U(PowerAction, <, PowerActionShutdown);
-
     if (DeviceState == __FdoGetDevicePowerState(Fdo)) {
         IoSkipCurrentIrpStackLocation(Irp);
         status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
@@ -2270,8 +2352,9 @@ __FdoSetDevicePower(
              __FdoSetDevicePowerDown(Fdo, Irp);
 
 done:
-    Trace("<==== (%s:%s)(%08x)\n",
-          PowerDeviceStateName(DeviceState), 
+    Trace("%s: <==== (%s:%s)(%08x)\n",
+          __FdoGetName(Fdo),
+          PowerDeviceStateName(DeviceState),
           PowerActionName(PowerAction),
           status);
     return status;
@@ -2279,8 +2362,8 @@ done:
 
 __drv_functionClass(REQUEST_POWER_COMPLETE)
 __drv_sameIRQL
-VOID
-__FdoRequestSetDevicePower(
+static VOID
+FdoRequestSetDevicePowerUpComplete(
     IN  PDEVICE_OBJECT      DeviceObject,
     IN  UCHAR               MinorFunction,
     IN  POWER_STATE         PowerState,
@@ -2288,47 +2371,123 @@ __FdoRequestSetDevicePower(
     IN  PIO_STATUS_BLOCK    IoStatus
     )
 {
-    PKEVENT                 Event = Context;
+    PIRP                    Irp = (PIRP) Context;
 
     UNREFERENCED_PARAMETER(DeviceObject);
     UNREFERENCED_PARAMETER(MinorFunction);
     UNREFERENCED_PARAMETER(PowerState);
+    UNREFERENCED_PARAMETER(IoStatus);
 
-    ASSERT(NT_SUCCESS(IoStatus->Status));
-
-    KeSetEvent(Event, IO_NO_INCREMENT, FALSE);
+    IoCompleteRequest(Irp, IO_NO_INCREMENT);
 }
 
+__drv_functionClass(IO_WORKITEM_ROUTINE)
+__drv_sameIRQL
 static VOID
-FdoRequestSetDevicePower(
-    IN  PXENVIF_FDO         Fdo,
-    IN  DEVICE_POWER_STATE  DeviceState
+FdoSystemPowerUpWorker(
+    IN  PDEVICE_OBJECT  DeviceObject,
+    IN  PVOID           Context
     )
 {
-    POWER_STATE             PowerState;
-    KEVENT                  Event;
-    NTSTATUS                status;
+    PXENVIF_FDO         Fdo = (PXENVIF_FDO) Context;
+    PIRP                Irp;
+    PIO_STACK_LOCATION  StackLocation;
+    SYSTEM_POWER_STATE  SystemState;
+    POWER_STATE         PowerState;
+    NTSTATUS            status;
+
+    UNREFERENCED_PARAMETER(DeviceObject);
 
-    Trace("%s\n", PowerDeviceStateName(DeviceState));
+    Irp = InterlockedExchangePointer(&Fdo->SystemPowerIrp, NULL);
+    ASSERT(Irp != NULL);
 
-    ASSERT3U(KeGetCurrentIrql(), ==, PASSIVE_LEVEL);
+    StackLocation = IoGetCurrentIrpStackLocation(Irp);;
+    SystemState = StackLocation->Parameters.Power.State.SystemState;
 
-    PowerState.DeviceState = DeviceState;
-    KeInitializeEvent(&Event, NotificationEvent, FALSE);
+    Info("%s: %s -> %s\n",
+         __FdoGetName(Fdo),
+         PowerSystemStateName(__FdoGetSystemPowerState(Fdo)),
+         PowerSystemStateName(SystemState));
+
+    __FdoSetSystemPowerState(Fdo, PowerSystemHibernate);
+    FdoS4ToS3(Fdo);
+    __FdoSetSystemPowerState(Fdo, SystemState);
+
+    PowerState.DeviceState = Fdo->LowerDeviceCapabilities.DeviceState[SystemState];
 
     status = PoRequestPowerIrp(Fdo->LowerDeviceObject,
                                IRP_MN_SET_POWER,
                                PowerState,
-                               __FdoRequestSetDevicePower,
-                               &Event,
+                               FdoRequestSetDevicePowerUpComplete,
+                               Irp,
                                NULL);
-    ASSERT(NT_SUCCESS(status));
 
-    (VOID) KeWaitForSingleObject(&Event,
-                                 Executive,
-                                 KernelMode,
-                                 FALSE,
-                                 NULL);
+    if (!NT_SUCCESS(status))
+        goto fail1;
+
+    return;
+
+fail1:
+    Error("fail1 - but continue IRP processing. (%08x)\n", status);
+    IoCompleteRequest(Irp, IO_NO_INCREMENT);
+}
+
+__drv_functionClass(IO_COMPLETION_ROUTINE)
+__drv_sameIRQL
+static NTSTATUS
+FdoSetSystemPowerUpComplete(
+    IN  PDEVICE_OBJECT  DeviceObject,
+    IN  PIRP            Irp,
+    IN  PVOID           Context
+    )
+{
+    PXENVIF_FDO         Fdo = (PXENVIF_FDO) Context;
+    PIO_STACK_LOCATION  StackLocation;
+    SYSTEM_POWER_STATE  SystemState;
+    POWER_STATE         PowerState;
+    NTSTATUS            status;
+
+    UNREFERENCED_PARAMETER(DeviceObject);
+
+    StackLocation = IoGetCurrentIrpStackLocation(Irp);
+    SystemState = StackLocation->Parameters.Power.State.SystemState;
+
+    if (SystemState < PowerSystemHibernate &&
+        __FdoGetSystemPowerState(Fdo) >= PowerSystemHibernate) {
+        PVOID           Exchange;
+
+        Exchange = InterlockedExchangePointer(&Fdo->SystemPowerIrp, Irp);
+        ASSERT(Exchange == NULL);
+
+        IoQueueWorkItem(Fdo->SystemPowerWorkItem,
+                        FdoSystemPowerUpWorker,
+                        DelayedWorkQueue,
+                        Fdo);
+    } else {
+        Info("%s: %s -> %s\n",
+             __FdoGetName(Fdo),
+             PowerSystemStateName(__FdoGetSystemPowerState(Fdo)),
+             PowerSystemStateName(SystemState));
+
+        __FdoSetSystemPowerState(Fdo, SystemState);
+
+        PowerState.DeviceState = Fdo->LowerDeviceCapabilities.DeviceState[SystemState];
+
+        status = PoRequestPowerIrp(Fdo->LowerDeviceObject,
+                                   IRP_MN_SET_POWER,
+                                   PowerState,
+                                   FdoRequestSetDevicePowerUpComplete,
+                                   Irp,
+                                   NULL);
+        if (!NT_SUCCESS(status))
+            goto fail1;
+    }
+
+    return STATUS_MORE_PROCESSING_REQUIRED;
+
+fail1:
+    Error("fail1 - but continue IRP processing. (%08x)\n", status);
+    return STATUS_CONTINUE_COMPLETION;
 }
 
 static FORCEINLINE NTSTATUS
@@ -2337,42 +2496,103 @@ __FdoSetSystemPowerUp(
     IN  PIRP            Irp
     )
 {
+    PIO_STACK_LOCATION  StackLocation;
+    SYSTEM_POWER_STATE  SystemState;
+
+    StackLocation = IoGetCurrentIrpStackLocation(Irp);
+    SystemState = StackLocation->Parameters.Power.State.SystemState;
 
+    ASSERT3U(SystemState, <, __FdoGetSystemPowerState(Fdo));
+
+    IoCopyCurrentIrpStackLocationToNext(Irp);
+    IoSetCompletionRoutine(Irp,
+                           FdoSetSystemPowerUpComplete,
+                           Fdo,
+                           TRUE,
+                           TRUE,
+                           TRUE);
+
+    IoCallDriver(Fdo->LowerDeviceObject, Irp);
+
+    return STATUS_PENDING;
+}
+
+__drv_functionClass(IO_WORKITEM_ROUTINE)
+__drv_sameIRQL
+static VOID
+FdoSystemPowerDownWorker(
+    IN  PDEVICE_OBJECT  DeviceObject,
+    IN  PVOID           Context
+    )
+{
+    PXENVIF_FDO         Fdo = (PXENVIF_FDO) Context;
+    PIRP                Irp;
     PIO_STACK_LOCATION  StackLocation;
     SYSTEM_POWER_STATE  SystemState;
-    DEVICE_POWER_STATE  DeviceState;
-    NTSTATUS            status;
+
+    UNREFERENCED_PARAMETER(DeviceObject);
+
+    Irp = InterlockedExchangePointer(&Fdo->SystemPowerIrp, NULL);
+    ASSERT(Irp != NULL);
 
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
     SystemState = StackLocation->Parameters.Power.State.SystemState;
 
-    ASSERT3U(SystemState, <,  __FdoGetSystemPowerState(Fdo));
+    __FdoSetSystemPowerState(Fdo, PowerSystemSleeping3);
+    FdoS3ToS4(Fdo);
+    __FdoSetSystemPowerState(Fdo, SystemState);
 
-    status = FdoForwardIrpSynchronously(Fdo, Irp);
-    if (!NT_SUCCESS(status))
-        goto done;
+    IoCopyCurrentIrpStackLocationToNext(Irp);
+    IoCallDriver(Fdo->LowerDeviceObject, Irp);
+}
+
+__drv_functionClass(REQUEST_POWER_COMPLETE)
+__drv_sameIRQL
+static VOID
+FdoRequestSetDevicePowerDownComplete(
+    IN  PDEVICE_OBJECT      DeviceObject,
+    IN  UCHAR               MinorFunction,
+    IN  POWER_STATE         PowerState,
+    IN  PVOID               Context,
+    IN  PIO_STATUS_BLOCK    IoStatus
+    )
+{
+    PIRP                    Irp = (PIRP) Context;
+    PIO_STACK_LOCATION      StackLocation = IoGetCurrentIrpStackLocation(Irp);
+    PDEVICE_OBJECT          UpperDeviceObject = StackLocation->DeviceObject;
+    PXENVIF_DX              Dx = (PXENVIF_DX)UpperDeviceObject->DeviceExtension;
+    PXENVIF_FDO             Fdo = Dx->Fdo;
+    SYSTEM_POWER_STATE      SystemState = StackLocation->Parameters.Power.State.SystemState;
+
+    UNREFERENCED_PARAMETER(DeviceObject);
+    UNREFERENCED_PARAMETER(MinorFunction);
+    UNREFERENCED_PARAMETER(PowerState);
+
+    if (!NT_SUCCESS(IoStatus->Status))
+        Error("fail1 - but continue IRP processing. (%08x)\n", IoStatus->Status);
 
     Info("%s: %s -> %s\n",
          __FdoGetName(Fdo),
-         PowerSystemStateName(__FdoGetSystemPowerState(Fdo)),
-         PowerSystemStateName(SystemState));
+        PowerSystemStateName(__FdoGetSystemPowerState(Fdo)),
+        PowerSystemStateName(SystemState));
 
-    if (SystemState < PowerSystemHibernate &&
-        __FdoGetSystemPowerState(Fdo) >= PowerSystemHibernate) {
-        __FdoSetSystemPowerState(Fdo, PowerSystemHibernate);
-        FdoS4ToS3(Fdo);
-    }
-
-    __FdoSetSystemPowerState(Fdo, SystemState);
+    if (SystemState >= PowerSystemHibernate &&
+        __FdoGetSystemPowerState(Fdo) < PowerSystemHibernate) {
+        PVOID Exchange;
 
-    DeviceState = Fdo->LowerDeviceCapabilities.DeviceState[SystemState];
-    FdoRequestSetDevicePower(Fdo, DeviceState);
+        Exchange = InterlockedExchangePointer(&Fdo->SystemPowerIrp, Irp);
+        ASSERT(Exchange == NULL);
 
-done:
-    Irp->IoStatus.Status = status;
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
+        IoQueueWorkItem(Fdo->DevicePowerWorkItem,
+                        FdoSystemPowerDownWorker,
+                        DelayedWorkQueue,
+                        Fdo);
+    } else {
+        __FdoSetSystemPowerState(Fdo, SystemState);
 
-    return status;
+        IoCopyCurrentIrpStackLocationToNext(Irp);
+        IoCallDriver(Fdo->LowerDeviceObject, Irp);
+    }
 }
 
 static FORCEINLINE NTSTATUS
@@ -2383,33 +2603,38 @@ __FdoSetSystemPowerDown(
 {
     PIO_STACK_LOCATION  StackLocation;
     SYSTEM_POWER_STATE  SystemState;
-    DEVICE_POWER_STATE  DeviceState;
+    POWER_STATE         PowerState;
     NTSTATUS            status;
 
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
+    BUG_ON(StackLocation->DeviceObject != Fdo->Dx->DeviceObject);
     SystemState = StackLocation->Parameters.Power.State.SystemState;
 
-    ASSERT3U(SystemState, >,  __FdoGetSystemPowerState(Fdo));
-
-    DeviceState = Fdo->LowerDeviceCapabilities.DeviceState[SystemState];
-
-    FdoRequestSetDevicePower(Fdo, DeviceState);
+    ASSERT3U(SystemState, >, __FdoGetSystemPowerState(Fdo));
 
-    Info("%s: %s -> %s\n",
-         __FdoGetName(Fdo),
-         PowerSystemStateName(__FdoGetSystemPowerState(Fdo)),
-         PowerSystemStateName(SystemState));
+    PowerState.DeviceState = Fdo->LowerDeviceCapabilities.DeviceState[SystemState];
 
-    if (SystemState >= PowerSystemHibernate &&
-        __FdoGetSystemPowerState(Fdo) < PowerSystemHibernate) {
-        __FdoSetSystemPowerState(Fdo, PowerSystemSleeping3);
-        FdoS3ToS4(Fdo);
+    if (SystemState >= PowerSystemShutdown) {
+        IoCopyCurrentIrpStackLocationToNext(Irp);
+        status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
+    } else {
+        status = PoRequestPowerIrp(Fdo->LowerDeviceObject,
+                                   IRP_MN_SET_POWER,
+                                   PowerState,
+                                   FdoRequestSetDevicePowerDownComplete,
+                                   Irp,
+                                   NULL);
+        if (!NT_SUCCESS(status))
+            goto fail1;
     }
 
-    __FdoSetSystemPowerState(Fdo, SystemState);
+    return status;
 
-    IoSkipCurrentIrpStackLocation(Irp);
-    status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
+fail1:
+    Error("fail1 - but continue IRP processing. (%08x)\n", status);
+
+    IoCopyCurrentIrpStackLocationToNext(Irp);
+    IoCallDriver(Fdo->LowerDeviceObject, Irp);
 
     return status;
 }
@@ -2429,29 +2654,32 @@ __FdoSetSystemPower(
     SystemState = StackLocation->Parameters.Power.State.SystemState;
     PowerAction = StackLocation->Parameters.Power.ShutdownType;
 
-    Trace("====> (%s:%s)\n",
-          PowerSystemStateName(SystemState), 
-          PowerActionName(PowerAction));
+    IoMarkIrpPending(Irp);
 
-    ASSERT3U(PowerAction, <, PowerActionShutdown);
+    Trace("%s: ====> (%s:%s)\n",
+          __FdoGetName(Fdo),
+          PowerSystemStateName(SystemState),
+          PowerActionName(PowerAction));
 
     if (SystemState == __FdoGetSystemPowerState(Fdo)) {
-        IoSkipCurrentIrpStackLocation(Irp);
+        IoCopyCurrentIrpStackLocationToNext(Irp);
         status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
 
         goto done;
     }
 
-    status = (SystemState < __FdoGetSystemPowerState(Fdo)) ?
+    status = SystemState < __FdoGetSystemPowerState(Fdo) ?
              __FdoSetSystemPowerUp(Fdo, Irp) :
              __FdoSetSystemPowerDown(Fdo, Irp);
 
 done:
-    Trace("<==== (%s:%s)(%08x)\n",
-          PowerSystemStateName(SystemState), 
+    Trace("%s: <==== (%s:%s)(%08x)\n",
+          __FdoGetName(Fdo),
+          PowerSystemStateName(SystemState),
           PowerActionName(PowerAction),
           status);
-    return status;
+
+    return STATUS_PENDING;
 }
 
 static FORCEINLINE NTSTATUS
@@ -2467,12 +2695,10 @@ __FdoQueryDevicePowerUp(
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
     DeviceState = StackLocation->Parameters.Power.State.DeviceState;
 
-    ASSERT3U(DeviceState, <,  __FdoGetDevicePowerState(Fdo));
-
-    status = FdoForwardIrpSynchronously(Fdo, Irp);
+    ASSERT3U(DeviceState, <, __FdoGetDevicePowerState(Fdo));
 
-    Irp->IoStatus.Status = status;
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
+    IoSkipCurrentIrpStackLocation(Irp);
+    status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
 
     return status;
 }
@@ -2490,7 +2716,7 @@ __FdoQueryDevicePowerDown(
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
     DeviceState = StackLocation->Parameters.Power.State.DeviceState;
 
-    ASSERT3U(DeviceState, >,  __FdoGetDevicePowerState(Fdo));
+    ASSERT3U(DeviceState, >, __FdoGetDevicePowerState(Fdo));
 
     IoSkipCurrentIrpStackLocation(Irp);
     status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
@@ -2513,12 +2739,11 @@ __FdoQueryDevicePower(
     DeviceState = StackLocation->Parameters.Power.State.DeviceState;
     PowerAction = StackLocation->Parameters.Power.ShutdownType;
 
-    Trace("====> (%s:%s)\n",
-          PowerDeviceStateName(DeviceState), 
+    Trace("%s: ====> (%s:%s)\n",
+          __FdoGetName(Fdo),
+          PowerDeviceStateName(DeviceState),
           PowerActionName(PowerAction));
 
-    ASSERT3U(PowerAction, <, PowerActionShutdown);
-
     if (DeviceState == __FdoGetDevicePowerState(Fdo)) {
         IoSkipCurrentIrpStackLocation(Irp);
         status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
@@ -2531,8 +2756,9 @@ __FdoQueryDevicePower(
              __FdoQueryDevicePowerDown(Fdo, Irp);
 
 done:
-    Trace("<==== (%s:%s)(%08x)\n",
-          PowerDeviceStateName(DeviceState), 
+    Trace("%s: <==== (%s:%s)(%08x)\n",
+          __FdoGetName(Fdo),
+          PowerDeviceStateName(DeviceState),
           PowerActionName(PowerAction),
           status);
     return status;
@@ -2540,8 +2766,8 @@ done:
 
 __drv_functionClass(REQUEST_POWER_COMPLETE)
 __drv_sameIRQL
-VOID
-__FdoRequestQueryDevicePower(
+static VOID
+FdoRequestQueryDevicePowerUpComplete(
     IN  PDEVICE_OBJECT      DeviceObject,
     IN  UCHAR               MinorFunction,
     IN  POWER_STATE         PowerState,
@@ -2549,47 +2775,55 @@ __FdoRequestQueryDevicePower(
     IN  PIO_STATUS_BLOCK    IoStatus
     )
 {
-    PKEVENT                 Event = Context;
+    PIRP                    Irp = (PIRP) Context;
 
     UNREFERENCED_PARAMETER(DeviceObject);
     UNREFERENCED_PARAMETER(MinorFunction);
     UNREFERENCED_PARAMETER(PowerState);
 
-    ASSERT(NT_SUCCESS(IoStatus->Status));
+    if (!NT_SUCCESS(IoStatus->Status))
+        Irp->IoStatus.Status = IoStatus->Status;
 
-    KeSetEvent(Event, IO_NO_INCREMENT, FALSE);
+    IoCompleteRequest(Irp, IO_NO_INCREMENT);
 }
 
-static VOID
-FdoRequestQueryDevicePower(
-    IN  PXENVIF_FDO         Fdo,
-    IN  DEVICE_POWER_STATE  DeviceState
+__drv_functionClass(IO_COMPLETION_ROUTINE)
+__drv_sameIRQL
+static NTSTATUS
+FdoQuerySystemPowerUpComplete(
+    IN  PDEVICE_OBJECT  DeviceObject,
+    IN  PIRP            Irp,
+    IN  PVOID           Context
     )
 {
-    POWER_STATE             PowerState;
-    KEVENT                  Event;
-    NTSTATUS                status;
-
-    Trace("%s\n", PowerDeviceStateName(DeviceState));
+    PXENVIF_FDO         Fdo = (PXENVIF_FDO) Context;
+    PIO_STACK_LOCATION  StackLocation;
+    SYSTEM_POWER_STATE  SystemState;
+    POWER_STATE         PowerState;
+    NTSTATUS            status;
 
-    ASSERT3U(KeGetCurrentIrql(), ==, PASSIVE_LEVEL);
+    UNREFERENCED_PARAMETER(DeviceObject);
 
-    PowerState.DeviceState = DeviceState;
-    KeInitializeEvent(&Event, NotificationEvent, FALSE);
+    StackLocation = IoGetCurrentIrpStackLocation(Irp);
+    SystemState = StackLocation->Parameters.Power.State.SystemState;
+    PowerState.DeviceState = Fdo->LowerDeviceCapabilities.DeviceState[SystemState];
 
     status = PoRequestPowerIrp(Fdo->LowerDeviceObject,
                                IRP_MN_QUERY_POWER,
                                PowerState,
-                               __FdoRequestQueryDevicePower,
-                               &Event,
+                               FdoRequestQueryDevicePowerUpComplete,
+                               Irp,
                                NULL);
-    ASSERT(NT_SUCCESS(status));
+    if (!NT_SUCCESS(status))
+        goto fail1;
 
-    (VOID) KeWaitForSingleObject(&Event,
-                                 Executive,
-                                 KernelMode,
-                                 FALSE,
-                                 NULL);
+    return STATUS_MORE_PROCESSING_REQUIRED;
+
+fail1:
+    Error("fail1 (%08x)\n", status);
+    Irp->IoStatus.Status = status;
+
+    return STATUS_CONTINUE_COMPLETION;
 }
 
 static FORCEINLINE NTSTATUS
@@ -2598,30 +2832,61 @@ __FdoQuerySystemPowerUp(
     IN  PIRP            Irp
     )
 {
-
     PIO_STACK_LOCATION  StackLocation;
     SYSTEM_POWER_STATE  SystemState;
-    DEVICE_POWER_STATE  DeviceState;
-    NTSTATUS            status;
 
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
     SystemState = StackLocation->Parameters.Power.State.SystemState;
 
-    ASSERT3U(SystemState, <,  __FdoGetSystemPowerState(Fdo));
+    ASSERT3U(SystemState, <, __FdoGetSystemPowerState(Fdo));
 
-    status = FdoForwardIrpSynchronously(Fdo, Irp);
-    if (!NT_SUCCESS(status))
-        goto done;
+    IoMarkIrpPending(Irp);
+    IoCopyCurrentIrpStackLocationToNext(Irp);
+    IoSetCompletionRoutine(Irp,
+                           FdoQuerySystemPowerUpComplete,
+                           Fdo,
+                           TRUE,
+                           TRUE,
+                           TRUE);
+    (VOID) IoCallDriver(Fdo->LowerDeviceObject, Irp);
 
-    DeviceState = Fdo->LowerDeviceCapabilities.DeviceState[SystemState];
+    return STATUS_PENDING;
+}
 
-    FdoRequestQueryDevicePower(Fdo, DeviceState);
+__drv_functionClass(REQUEST_POWER_COMPLETE)
+__drv_sameIRQL
+static VOID
+FdoRequestQueryDevicePowerDownComplete(
+    IN  PDEVICE_OBJECT      DeviceObject,
+    IN  UCHAR               MinorFunction,
+    IN  POWER_STATE         PowerState,
+    IN  PVOID               Context,
+    IN  PIO_STATUS_BLOCK    IoStatus
+    )
+{
+    PIRP                    Irp = (PIRP) Context;
+    PIO_STACK_LOCATION      StackLocation = IoGetCurrentIrpStackLocation(Irp);
+    PDEVICE_OBJECT          UpperDeviceObject = StackLocation->DeviceObject;
+    PXENVIF_DX              Dx = (PXENVIF_DX)UpperDeviceObject->DeviceExtension;
+    PXENVIF_FDO             Fdo = Dx->Fdo;
 
-done:
-    Irp->IoStatus.Status = status;
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
+    UNREFERENCED_PARAMETER(DeviceObject);
+    UNREFERENCED_PARAMETER(MinorFunction);
+    UNREFERENCED_PARAMETER(PowerState);
 
-    return status;
+    if (!NT_SUCCESS(IoStatus->Status))
+        goto fail1;
+
+    IoCopyCurrentIrpStackLocationToNext(Irp);
+    IoCallDriver(Fdo->LowerDeviceObject, Irp);
+
+    return;
+
+fail1:
+    Error("fail1 (%08x)\n", IoStatus->Status);
+
+    Irp->IoStatus.Status = IoStatus->Status;
+    IoCompleteRequest(Irp, IO_NO_INCREMENT);
 }
 
 static FORCEINLINE NTSTATUS
@@ -2632,7 +2897,7 @@ __FdoQuerySystemPowerDown(
 {
     PIO_STACK_LOCATION  StackLocation;
     SYSTEM_POWER_STATE  SystemState;
-    DEVICE_POWER_STATE  DeviceState;
+    POWER_STATE         PowerState;
     NTSTATUS            status;
 
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
@@ -2640,12 +2905,26 @@ __FdoQuerySystemPowerDown(
 
     ASSERT3U(SystemState, >,  __FdoGetSystemPowerState(Fdo));
 
-    DeviceState = Fdo->LowerDeviceCapabilities.DeviceState[SystemState];
+    PowerState.DeviceState = Fdo->LowerDeviceCapabilities.DeviceState[SystemState];
+
+    status = PoRequestPowerIrp(Fdo->LowerDeviceObject,
+                               IRP_MN_QUERY_POWER,
+                               PowerState,
+                               FdoRequestQueryDevicePowerDownComplete,
+                               Irp,
+                               NULL);
+    if (!NT_SUCCESS(status))
+        goto fail1;
+
+    IoMarkIrpPending(Irp);
 
-    FdoRequestQueryDevicePower(Fdo, DeviceState);
+    return STATUS_PENDING;
 
-    IoSkipCurrentIrpStackLocation(Irp);
-    status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
+fail1:
+    Error("fail1 (%08x)\n", status);
+
+    Irp->IoStatus.Status = status;
+    IoCompleteRequest(Irp, IO_NO_INCREMENT);
 
     return status;
 }
@@ -2665,12 +2944,11 @@ __FdoQuerySystemPower(
     SystemState = StackLocation->Parameters.Power.State.SystemState;
     PowerAction = StackLocation->Parameters.Power.ShutdownType;
 
-    Trace("====> (%s:%s)\n",
-          PowerSystemStateName(SystemState), 
+    Trace("%s: ====> (%s:%s)\n",
+          __FdoGetName(Fdo),
+          PowerSystemStateName(SystemState),
           PowerActionName(PowerAction));
 
-    ASSERT3U(PowerAction, <, PowerActionShutdown);
-
     if (SystemState == __FdoGetSystemPowerState(Fdo)) {
         IoSkipCurrentIrpStackLocation(Irp);
         status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
@@ -2683,184 +2961,93 @@ __FdoQuerySystemPower(
              __FdoQuerySystemPowerDown(Fdo, Irp);
 
 done:
-    Trace("<==== (%s:%s)(%08x)\n",
-          PowerSystemStateName(SystemState), 
+    Trace("%s: <==== (%s:%s)(%08x)\n",
+          __FdoGetName(Fdo),
+          PowerSystemStateName(SystemState),
           PowerActionName(PowerAction),
           status);
 
     return status;
 }
 
-static NTSTATUS
-FdoDevicePower(
-    IN  PXENVIF_THREAD  Self,
-    IN  PVOID           Context
+static FORCEINLINE NTSTATUS
+__FdoDevicePower(
+    IN  PXENVIF_FDO     Fdo,
+    IN  PIRP            Irp
     )
 {
-    PXENVIF_FDO         Fdo = Context;
-    PKEVENT             Event;
-
-    Event = ThreadGetEvent(Self);
-
-    for (;;) {
-        PIRP                Irp;
-        PIO_STACK_LOCATION  StackLocation;
-        UCHAR               MinorFunction;
-
-        if (Fdo->DevicePowerIrp == NULL) {
-            (VOID) KeWaitForSingleObject(Event,
-                                         Executive,
-                                         KernelMode,
-                                         FALSE,
-                                         NULL);
-            KeClearEvent(Event);
-        }
-
-        if (ThreadIsAlerted(Self))
-            break;
-
-        Irp = Fdo->DevicePowerIrp;
-
-        if (Irp == NULL)
-            continue;
-
-        Fdo->DevicePowerIrp = NULL;
-        KeMemoryBarrier();
+    PIO_STACK_LOCATION  StackLocation;
+    NTSTATUS            status;
 
-        StackLocation = IoGetCurrentIrpStackLocation(Irp);
-        MinorFunction = StackLocation->MinorFunction;
+    StackLocation = IoGetCurrentIrpStackLocation(Irp);
 
-        switch (StackLocation->MinorFunction) {
-        case IRP_MN_SET_POWER:
-            (VOID) __FdoSetDevicePower(Fdo, Irp);
-            break;
+    switch (StackLocation->MinorFunction) {
+    case IRP_MN_SET_POWER:
+        status = __FdoSetDevicePower(Fdo, Irp);
+        break;
 
-        case IRP_MN_QUERY_POWER:
-            (VOID) __FdoQueryDevicePower(Fdo, Irp);
-            break;
+    case IRP_MN_QUERY_POWER:
+        status = __FdoQueryDevicePower(Fdo, Irp);
+        break;
 
-        default:
-            ASSERT(FALSE);
-            break;
-        }
+    default:
+        IoSkipCurrentIrpStackLocation(Irp);
+        status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
+        break;
     }
 
-    return STATUS_SUCCESS;
+    return status;
 }
 
-static NTSTATUS
-FdoSystemPower(
-    IN  PXENVIF_THREAD  Self,
-    IN  PVOID           Context
+static FORCEINLINE NTSTATUS
+__FdoSystemPower(
+    IN  PXENVIF_FDO     Fdo,
+    IN  PIRP            Irp
     )
 {
-    PXENVIF_FDO         Fdo = Context;
-    PKEVENT             Event;
-
-    Event = ThreadGetEvent(Self);
-
-    for (;;) {
-        PIRP                Irp;
-        PIO_STACK_LOCATION  StackLocation;
-        UCHAR               MinorFunction;
-
-        if (Fdo->SystemPowerIrp == NULL) {
-            (VOID) KeWaitForSingleObject(Event,
-                                         Executive,
-                                         KernelMode,
-                                         FALSE,
-                                         NULL);
-            KeClearEvent(Event);
-        }
-
-        if (ThreadIsAlerted(Self))
-            break;
-
-        Irp = Fdo->SystemPowerIrp;
-
-        if (Irp == NULL)
-            continue;
-
-        Fdo->SystemPowerIrp = NULL;
-        KeMemoryBarrier();
+    PIO_STACK_LOCATION  StackLocation;
+    NTSTATUS            status;
 
-        StackLocation = IoGetCurrentIrpStackLocation(Irp);
-        MinorFunction = StackLocation->MinorFunction;
+    StackLocation = IoGetCurrentIrpStackLocation(Irp);
 
-        switch (StackLocation->MinorFunction) {
-        case IRP_MN_SET_POWER:
-            (VOID) __FdoSetSystemPower(Fdo, Irp);
-            break;
+    switch (StackLocation->MinorFunction) {
+    case IRP_MN_SET_POWER:
+        status = __FdoSetSystemPower(Fdo, Irp);
+        break;
 
-        case IRP_MN_QUERY_POWER:
-            (VOID) __FdoQuerySystemPower(Fdo, Irp);
-            break;
+    case IRP_MN_QUERY_POWER:
+        status = __FdoQuerySystemPower(Fdo, Irp);
+        break;
 
-        default:
-            ASSERT(FALSE);
-            break;
-        }
+    default:
+        IoSkipCurrentIrpStackLocation(Irp);
+        status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
+        break;
     }
 
-    return STATUS_SUCCESS;
+    return status;
 }
 
 static DECLSPEC_NOINLINE NTSTATUS
 FdoDispatchPower(
-    IN  PXENVIF_FDO     Fdo,
+    IN  PXENVIF_FDO   Fdo,
     IN  PIRP            Irp
     )
 {
     PIO_STACK_LOCATION  StackLocation;
-    UCHAR               MinorFunction;
     POWER_STATE_TYPE    PowerType;
-    POWER_ACTION        PowerAction;
     NTSTATUS            status;
 
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    MinorFunction = StackLocation->MinorFunction;
-
-    if (MinorFunction != IRP_MN_QUERY_POWER &&
-        MinorFunction != IRP_MN_SET_POWER) {
-        IoSkipCurrentIrpStackLocation(Irp);
-        status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
-
-        goto done;
-    }
-
     PowerType = StackLocation->Parameters.Power.Type;
-    PowerAction = StackLocation->Parameters.Power.ShutdownType;
-
-    if (PowerAction >= PowerActionShutdown) {
-        IoSkipCurrentIrpStackLocation(Irp);
-        status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
-
-        goto done;
-    }
 
     switch (PowerType) {
     case DevicePowerState:
-        IoMarkIrpPending(Irp);
-
-        ASSERT3P(Fdo->DevicePowerIrp, ==, NULL);
-        Fdo->DevicePowerIrp = Irp;
-        KeMemoryBarrier();
-
-        ThreadWake(Fdo->DevicePowerThread);
-
-        status = STATUS_PENDING;
+        status = __FdoDevicePower(Fdo, Irp);
         break;
 
     case SystemPowerState:
-        IoMarkIrpPending(Irp);
-
-        ASSERT3P(Fdo->SystemPowerIrp, ==, NULL);
-        Fdo->SystemPowerIrp = Irp;
-        KeMemoryBarrier();
-
-        ThreadWake(Fdo->SystemPowerThread);
-
-        status = STATUS_PENDING;
+        status = __FdoSystemPower(Fdo, Irp);
         break;
 
     default:
@@ -2869,7 +3056,6 @@ FdoDispatchPower(
         break;
     }
 
-done:
     return status;
 }
 
@@ -3063,12 +3249,12 @@ FdoCreate(
     Fdo->LowerDeviceObject = IoAttachDeviceToDeviceStack(FunctionDeviceObject,
                                                          PhysicalDeviceObject);
 
-    status = ThreadCreate(FdoSystemPower, Fdo, &Fdo->SystemPowerThread);
-    if (!NT_SUCCESS(status))
+    Fdo->SystemPowerWorkItem = IoAllocateWorkItem(PhysicalDeviceObject);
+    if (Fdo->SystemPowerWorkItem == NULL)
         goto fail3;
 
-    status = ThreadCreate(FdoDevicePower, Fdo, &Fdo->DevicePowerThread);
-    if (!NT_SUCCESS(status))
+    Fdo->DevicePowerWorkItem = IoAllocateWorkItem(PhysicalDeviceObject);
+    if (Fdo->DevicePowerWorkItem == NULL)
         goto fail4;
 
     status = __FdoAcquireLowerBusInterface(Fdo);
@@ -3226,16 +3412,14 @@ fail6:
 fail5:
     Error("fail5\n");
 
-    ThreadAlert(Fdo->DevicePowerThread);
-    ThreadJoin(Fdo->DevicePowerThread);
-    Fdo->DevicePowerThread = NULL;
-    
+    IoFreeWorkItem(Fdo->DevicePowerWorkItem);
+    Fdo->DevicePowerWorkItem = NULL;
+
 fail4:
     Error("fail4\n");
 
-    ThreadAlert(Fdo->SystemPowerThread);
-    ThreadJoin(Fdo->SystemPowerThread);
-    Fdo->SystemPowerThread = NULL;
+    IoFreeWorkItem(Fdo->SystemPowerWorkItem);
+    Fdo->SystemPowerWorkItem = NULL;
     
 fail3:
     Error("fail3\n");
@@ -3311,13 +3495,11 @@ FdoDestroy(
 
     __FdoReleaseLowerBusInterface(Fdo);
 
-    ThreadAlert(Fdo->DevicePowerThread);
-    ThreadJoin(Fdo->DevicePowerThread);
-    Fdo->DevicePowerThread = NULL;
+    IoFreeWorkItem(Fdo->DevicePowerWorkItem);
+    Fdo->DevicePowerWorkItem = NULL;
 
-    ThreadAlert(Fdo->SystemPowerThread);
-    ThreadJoin(Fdo->SystemPowerThread);
-    Fdo->SystemPowerThread = NULL;
+    IoFreeWorkItem(Fdo->SystemPowerWorkItem);
+    Fdo->SystemPowerWorkItem = NULL;
 
     IoDetachDevice(Fdo->LowerDeviceObject);
 
diff --git a/src/xenvif/pdo.c b/src/xenvif/pdo.c
index 0cef2d5..2979f53 100644
--- a/src/xenvif/pdo.c
+++ b/src/xenvif/pdo.c
@@ -68,9 +68,9 @@
 struct _XENVIF_PDO {
     PXENVIF_DX                  Dx;
 
-    PXENVIF_THREAD              SystemPowerThread;
+    PIO_WORKITEM                SystemPowerWorkItem;
     PIRP                        SystemPowerIrp;
-    PXENVIF_THREAD              DevicePowerThread;
+    PIO_WORKITEM                DevicePowerWorkItem;
     PIRP                        DevicePowerIrp;
 
     PXENVIF_FDO                 Fdo;
@@ -2365,28 +2365,31 @@ PdoDispatchPnp(
     return status;
 }
 
-static FORCEINLINE NTSTATUS
-__PdoSetDevicePower(
-    IN  PXENVIF_PDO     Pdo,
-    IN  PIRP            Irp
+__drv_functionClass(IO_WORKITEM_ROUTINE)
+__drv_sameIRQL
+static VOID
+PdoDevicePowerWorker(
+    IN  PDEVICE_OBJECT  DeviceObject,
+    IN  PVOID           Context
     )
 {
+    PXENVIF_PDO         Pdo = (PXENVIF_PDO) Context;
+    PIRP                Irp;
     PIO_STACK_LOCATION  StackLocation;
     DEVICE_POWER_STATE  DeviceState;
     POWER_ACTION        PowerAction;
     NTSTATUS            status;
 
+    UNREFERENCED_PARAMETER(DeviceObject);
+
+    Irp = InterlockedExchangePointer(&Pdo->DevicePowerIrp, NULL);
+    ASSERT(Irp != NULL);
+
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
     DeviceState = StackLocation->Parameters.Power.State.DeviceState;
     PowerAction = StackLocation->Parameters.Power.ShutdownType;
 
-    Trace("====> (%s) (%s:%s)\n",
-          __PdoGetName(Pdo),
-          PowerDeviceStateName(DeviceState), 
-          PowerActionName(PowerAction));
-
-    ASSERT3U(PowerAction, <, PowerActionShutdown);
-
+    status = STATUS_SUCCESS;
     if (__PdoGetDevicePowerState(Pdo) > DeviceState) {
         Trace("%s: POWERING UP: %s -> %s\n",
               __PdoGetName(Pdo),
@@ -2395,7 +2398,6 @@ __PdoSetDevicePower(
 
         ASSERT3U(DeviceState, ==, PowerDeviceD0);
         status = PdoD3ToD0(Pdo);
-        ASSERT(NT_SUCCESS(status));
     } else if (__PdoGetDevicePowerState(Pdo) < DeviceState) {
         Trace("%s: POWERING DOWN: %s -> %s\n",
               __PdoGetName(Pdo),
@@ -2406,77 +2408,71 @@ __PdoSetDevicePower(
         PdoD0ToD3(Pdo);
     }
 
+    /* Cannot fail the IRP at this point, keep going. */
     Irp->IoStatus.Status = STATUS_SUCCESS;
     IoCompleteRequest(Irp, IO_NO_INCREMENT);
 
-    Trace("<==== (%s:%s)\n",
+    Trace("<==== (%s:%s)(%08x)\n",
           PowerDeviceStateName(DeviceState), 
-          PowerActionName(PowerAction));
-
-    return STATUS_SUCCESS;
+          PowerActionName(PowerAction),
+          status);
 }
 
-static NTSTATUS
-PdoDevicePower(
-    IN  PXENVIF_THREAD  Self,
-    IN  PVOID           Context
+static FORCEINLINE NTSTATUS
+__PdoSetDevicePower(
+    IN  PXENVIF_PDO     Pdo,
+    IN  PIRP            Irp
     )
 {
-    PXENVIF_PDO         Pdo = Context;
-    PKEVENT             Event;
-
-    Event = ThreadGetEvent(Self);
-
-    for (;;) {
-        PIRP    Irp;
-
-        if (Pdo->DevicePowerIrp == NULL) {
-            (VOID) KeWaitForSingleObject(Event,
-                                         Executive,
-                                         KernelMode,
-                                         FALSE,
-                                         NULL);
-            KeClearEvent(Event);
-        }
+    PIO_STACK_LOCATION  StackLocation;
+    DEVICE_POWER_STATE  DeviceState;
+    POWER_ACTION        PowerAction;
+    PVOID               Exchange;
 
-        if (ThreadIsAlerted(Self))
-            break;
+    StackLocation = IoGetCurrentIrpStackLocation(Irp);
+    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
+    PowerAction = StackLocation->Parameters.Power.ShutdownType;
 
-        Irp = Pdo->DevicePowerIrp;
+    Trace("====> (%s:%s)\n",
+          PowerDeviceStateName(DeviceState),
+          PowerActionName(PowerAction));
 
-        if (Irp == NULL)
-            continue;
+    IoMarkIrpPending(Irp);
 
-        Pdo->DevicePowerIrp = NULL;
-        KeMemoryBarrier();
+    Exchange = InterlockedExchangePointer(&Pdo->DevicePowerIrp, Irp);
+    ASSERT(Exchange == NULL);
 
-        (VOID) __PdoSetDevicePower(Pdo, Irp);
-    }
+    IoQueueWorkItem(Pdo->DevicePowerWorkItem,
+                    PdoDevicePowerWorker,
+                    DelayedWorkQueue,
+                    Pdo);
 
-    return STATUS_SUCCESS;
+    return STATUS_PENDING;
 }
 
-static FORCEINLINE NTSTATUS
-__PdoSetSystemPower(
-    IN  PXENVIF_PDO         Pdo,
-    IN  PIRP                Irp
+__drv_functionClass(IO_WORKITEM_ROUTINE)
+__drv_sameIRQL
+static VOID
+PdoSystemPowerWorker(
+    IN  PDEVICE_OBJECT  DeviceObject,
+    IN  PVOID           Context
     )
 {
-    PIO_STACK_LOCATION      StackLocation;
-    SYSTEM_POWER_STATE      SystemState;
-    POWER_ACTION            PowerAction;
+    PXENVIF_PDO         Pdo = (PXENVIF_PDO) Context;
+    PIRP                Irp;
+    PIO_STACK_LOCATION  StackLocation;
+    SYSTEM_POWER_STATE  SystemState;
+    POWER_ACTION        PowerAction;
+
+    UNREFERENCED_PARAMETER(DeviceObject);
+
+    Irp = InterlockedExchangePointer(&Pdo->SystemPowerIrp, NULL);
+    ASSERT(Irp != NULL);
 
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
     SystemState = StackLocation->Parameters.Power.State.SystemState;
     PowerAction = StackLocation->Parameters.Power.ShutdownType;
 
-    Trace("====> (%s) (%s:%s)\n",
-          __PdoGetName(Pdo),
-          PowerSystemStateName(SystemState), 
-          PowerActionName(PowerAction));
-
-    ASSERT3U(PowerAction, <, PowerActionShutdown);
-
     if (__PdoGetSystemPowerState(Pdo) > SystemState) {
         if (SystemState < PowerSystemHibernate &&
             __PdoGetSystemPowerState(Pdo) >= PowerSystemHibernate) {
@@ -2488,6 +2484,7 @@ __PdoSetSystemPower(
               __PdoGetName(Pdo),
               PowerSystemStateName(__PdoGetSystemPowerState(Pdo)),
               PowerSystemStateName(SystemState));
+
     } else if (__PdoGetSystemPowerState(Pdo) < SystemState) {
         Trace("%s: POWERING DOWN: %s -> %s\n",
               __PdoGetName(Pdo),
@@ -2509,97 +2506,60 @@ __PdoSetSystemPower(
     Trace("<==== (%s:%s)\n",
           PowerSystemStateName(SystemState), 
           PowerActionName(PowerAction));
-
-    return STATUS_SUCCESS;
 }
 
-static NTSTATUS
-PdoSystemPower(
-    IN  PXENVIF_THREAD  Self,
-    IN  PVOID           Context
+static FORCEINLINE NTSTATUS
+__PdoSetSystemPower(
+    IN  PXENVIF_PDO     Pdo,
+    IN  PIRP            Irp
     )
 {
-    PXENVIF_PDO         Pdo = Context;
-    PKEVENT             Event;
-
-    Event = ThreadGetEvent(Self);
-
-    for (;;) {
-        PIRP    Irp;
-
-        if (Pdo->SystemPowerIrp == NULL) {
-            (VOID) KeWaitForSingleObject(Event,
-                                         Executive,
-                                         KernelMode,
-                                         FALSE,
-                                         NULL);
-            KeClearEvent(Event);
-        }
+    PIO_STACK_LOCATION  StackLocation;
+    SYSTEM_POWER_STATE  SystemState;
+    POWER_ACTION        PowerAction;
+    PVOID               Exchange;
 
-        if (ThreadIsAlerted(Self))
-            break;
+    StackLocation = IoGetCurrentIrpStackLocation(Irp);
+    SystemState = StackLocation->Parameters.Power.State.SystemState;
+    PowerAction = StackLocation->Parameters.Power.ShutdownType;
 
-        Irp = Pdo->SystemPowerIrp;
+    Trace("====> (%s:%s)\n",
+          PowerSystemStateName(SystemState),
+          PowerActionName(PowerAction));
 
-        if (Irp == NULL)
-            continue;
+    IoMarkIrpPending(Irp);
 
-        Pdo->SystemPowerIrp = NULL;
-        KeMemoryBarrier();
+    Exchange = InterlockedExchangePointer(&Pdo->SystemPowerIrp, Irp);
+    ASSERT(Exchange == NULL);
 
-        (VOID) __PdoSetSystemPower(Pdo, Irp);
-    }
+    IoQueueWorkItem(Pdo->SystemPowerWorkItem,
+                    PdoSystemPowerWorker,
+                    DelayedWorkQueue,
+                    Pdo);
 
-    return STATUS_SUCCESS;
+    return STATUS_PENDING;
 }
 
-static DECLSPEC_NOINLINE NTSTATUS
-PdoSetPower(
+static FORCEINLINE NTSTATUS
+__PdoSetPower(
     IN  PXENVIF_PDO     Pdo,
     IN  PIRP            Irp
     )
 {
     PIO_STACK_LOCATION  StackLocation;
     POWER_STATE_TYPE    PowerType;
-    POWER_ACTION        PowerAction;
     NTSTATUS            status;
     
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
     PowerType = StackLocation->Parameters.Power.Type;
-    PowerAction = StackLocation->Parameters.Power.ShutdownType;
-
-    if (PowerAction >= PowerActionShutdown) {
-        Irp->IoStatus.Status = STATUS_SUCCESS;
-
-        status = Irp->IoStatus.Status;
-        IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
-        goto done;
-    }
 
     switch (PowerType) {
     case DevicePowerState:
-        IoMarkIrpPending(Irp);
-
-        ASSERT3P(Pdo->DevicePowerIrp, ==, NULL);
-        Pdo->DevicePowerIrp = Irp;
-        KeMemoryBarrier();
-
-        ThreadWake(Pdo->DevicePowerThread);
-
-        status = STATUS_PENDING;
+        status = __PdoSetDevicePower(Pdo, Irp);
         break;
 
     case SystemPowerState:
-        IoMarkIrpPending(Irp);
-
-        ASSERT3P(Pdo->SystemPowerIrp, ==, NULL);
-        Pdo->SystemPowerIrp = Irp;
-        KeMemoryBarrier();
-
-        ThreadWake(Pdo->SystemPowerThread);
-
-        status = STATUS_PENDING;
+        status = __PdoSetSystemPower(Pdo, Irp);
         break;
 
     default:
@@ -2608,48 +2568,41 @@ PdoSetPower(
         break;
     }
 
-done:    
     return status;
 }
 
-static DECLSPEC_NOINLINE NTSTATUS
-PdoQueryPower(
-    IN  PXENVIF_PDO     Pdo,
-    IN  PIRP            Irp
+static FORCEINLINE NTSTATUS
+__PdoQueryPower(
+    IN  PXENVIF_PDO Pdo,
+    IN  PIRP        Irp
     )
 {
-    NTSTATUS            status;
-    
     UNREFERENCED_PARAMETER(Pdo);
 
     Irp->IoStatus.Status = STATUS_SUCCESS;
-
-    status = Irp->IoStatus.Status;
     IoCompleteRequest(Irp, IO_NO_INCREMENT);
-    
-    return status;
+
+    return STATUS_SUCCESS;
 }
 
-static DECLSPEC_NOINLINE NTSTATUS
+static NTSTATUS
 PdoDispatchPower(
     IN  PXENVIF_PDO     Pdo,
     IN  PIRP            Irp
     )
 {
     PIO_STACK_LOCATION  StackLocation;
-    UCHAR               MinorFunction;
     NTSTATUS            status;
 
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    MinorFunction = StackLocation->MinorFunction;
 
     switch (StackLocation->MinorFunction) {
     case IRP_MN_SET_POWER:
-        status = PdoSetPower(Pdo, Irp);
+        status = __PdoSetPower(Pdo, Irp);
         break;
 
     case IRP_MN_QUERY_POWER:
-        status = PdoQueryPower(Pdo, Irp);
+        status = __PdoQueryPower(Pdo, Irp);
         break;
 
     default:
@@ -2762,12 +2715,12 @@ PdoCreate(
     Pdo->Dx = Dx;
     Pdo->Fdo = Fdo;
 
-    status = ThreadCreate(PdoSystemPower, Pdo, &Pdo->SystemPowerThread);
-    if (!NT_SUCCESS(status))
+    Pdo->SystemPowerWorkItem = IoAllocateWorkItem(PhysicalDeviceObject);
+    if (Pdo->SystemPowerWorkItem == NULL)
         goto fail3;
 
-    status = ThreadCreate(PdoDevicePower, Pdo, &Pdo->DevicePowerThread);
-    if (!NT_SUCCESS(status))
+    Pdo->DevicePowerWorkItem = IoAllocateWorkItem(PhysicalDeviceObject);
+    if (Pdo->DevicePowerWorkItem == NULL)
         goto fail4;
 
     __PdoSetName(Pdo, Number);
@@ -2860,16 +2813,14 @@ fail6:
 fail5:
     Error("fail5\n");
 
-    ThreadAlert(Pdo->DevicePowerThread);
-    ThreadJoin(Pdo->DevicePowerThread);
-    Pdo->DevicePowerThread = NULL;
+    IoFreeWorkItem(Pdo->DevicePowerWorkItem);
+    Pdo->DevicePowerWorkItem = NULL;
 
 fail4:
     Error("fail4\n");
 
-    ThreadAlert(Pdo->SystemPowerThread);
-    ThreadJoin(Pdo->SystemPowerThread);
-    Pdo->SystemPowerThread = NULL;
+    IoFreeWorkItem(Pdo->SystemPowerWorkItem);
+    Pdo->SystemPowerWorkItem = NULL;
 
 fail3:
     Error("fail3\n");
@@ -2939,13 +2890,11 @@ PdoDestroy(
 
     __PdoClearPermanentAddress(Pdo);
 
-    ThreadAlert(Pdo->DevicePowerThread);
-    ThreadJoin(Pdo->DevicePowerThread);
-    Pdo->DevicePowerThread = NULL;
+    IoFreeWorkItem(Pdo->DevicePowerWorkItem);
+    Pdo->DevicePowerWorkItem = NULL;
 
-    ThreadAlert(Pdo->SystemPowerThread);
-    ThreadJoin(Pdo->SystemPowerThread);
-    Pdo->SystemPowerThread = NULL;
+    IoFreeWorkItem(Pdo->SystemPowerWorkItem);
+    Pdo->SystemPowerWorkItem = NULL;
 
     Pdo->Fdo = NULL;
     Pdo->Dx = NULL;
-- 
2.41.0.windows.3



From win-pv-devel-bounces@lists.xenproject.org Mon Mar 25 09:28:09 2024
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Mon, 25 Mar 2024 09:28:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.697634.1088586 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rogch-00079w-7D; Mon, 25 Mar 2024 09:28:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 697634.1088586; Mon, 25 Mar 2024 09:28:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1rogch-00079o-40; Mon, 25 Mar 2024 09:28:07 +0000
Received: by outflank-mailman (input) for mailman id 697634;
 Mon, 25 Mar 2024 09:28:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JU+D=K7=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1rogcf-0006di-VO
 for win-pv-devel@lists.xenproject.org; Mon, 25 Mar 2024 09:28:06 +0000
Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com
 [2a00:1450:4864:20::12d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id faf20ead-ea89-11ee-a1ef-f123f15fe8a2;
 Mon, 25 Mar 2024 10:28:04 +0100 (CET)
Received: by mail-lf1-x12d.google.com with SMTP id
 2adb3069b0e04-515a86daf09so1459862e87.3
 for <win-pv-devel@lists.xenproject.org>; Mon, 25 Mar 2024 02:28:03 -0700 (PDT)
Received: from [192.168.21.70] (54-240-197-233.amazon.com. [54.240.197.233])
 by smtp.gmail.com with ESMTPSA id
 e7-20020a5d5947000000b0033f20497d4asm9033616wri.5.2024.03.25.02.28.01
 for <win-pv-devel@lists.xenproject.org>
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 25 Mar 2024 02:28:02 -0700 (PDT)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: faf20ead-ea89-11ee-a1ef-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1711358883; x=1711963683; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:organization:reply-to
         :content-language:references:to:subject:user-agent:mime-version:date
         :message-id:from:from:to:cc:subject:date:message-id:reply-to;
        bh=9JyW5C8g3fynXQpJpKkQPL8ConP3du9FWsnFIXMQL9g=;
        b=KRoTht8Rs7qyre+xQY//bPPYzthnB1czIuqWfUpOJmVypLM1pzbrVzuteOI2PlOzDA
         pSaNx19ezUWFKs7L/99FQGpmjOLvR5z/ffr1U4yfFWYFViKwT97ExYvpAtfjo2m1U75A
         gi3BOyWBRvPDFi53vNkq2PzzycI7tkXXQn9EyxYsdrB1c2NIZ++qTwkvgRizTBfUB9uf
         6I35CKFwUZpm6aPBShSz/Uhu60XKPkj4sEofbEBFw9G/P4PZ16Tlw2QpsZxWg74ynhP1
         gO01maOwzEOKMjzhWdw0iHWAYfbghtSHgqNQJxTXZCiQGNQhs1KVwcE3Xcc2pNG0rMp5
         E81w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1711358883; x=1711963683;
        h=content-transfer-encoding:in-reply-to:organization:reply-to
         :content-language:references:to:subject:user-agent:mime-version:date
         :message-id:from:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=9JyW5C8g3fynXQpJpKkQPL8ConP3du9FWsnFIXMQL9g=;
        b=FOmUZylOPhWGiPUaX4I3FZjy4VY/dZPHM81YycCK7RKbKrdIjc4VE+/H62F8LVp7Ce
         YMhtyxai43GRzea9qOmlMCC7dMwPxnA76Msz5UnzJKvNgWRWSMd6jnuPwOQLQJUaAZU1
         MH4VRi/pGKb1YQqqJljYO2DEyU+3Cozgf2yp2DtY7UDfMh0dTZI3zo1o/2+c0LHDdv/3
         mnv1Di6iGrmnoVVrgqL8+PLNeSmJYZidf4ju8P+RSfnL8XII3Rh6j2mZnNX1Sk1B0kcR
         fJrZVhqjzh7d4eOpLYm78HLTzj6/fvl52u8q/UJNw309PzYj3gVPr9o6hxvyNMzBIGrg
         i6xg==
X-Gm-Message-State: AOJu0YzlBAXSMXY4jzkzspgpla5rIynapVvewlCEJJgI+YXj7UWXHJM+
	IsgnTjPmLhFl1Pg7ud15G6fySI/wKsRIilBsiOE9FZQ1BqtUfwosiln3l1OGJoM=
X-Google-Smtp-Source: AGHT+IFrqzMT4ceqSVORutmO78h3wbJSeNDIWjSzHgm90/A4l2mJiGPJB6ur8zaVyL4Qt4t2FNH8yQ==
X-Received: by 2002:ac2:5bce:0:b0:513:e6b7:d75c with SMTP id u14-20020ac25bce000000b00513e6b7d75cmr4634209lfn.58.1711358883118;
        Mon, 25 Mar 2024 02:28:03 -0700 (PDT)
From: Paul Durrant <xadimgnik@gmail.com>
X-Google-Original-From: Paul Durrant <paul@xen.org>
Message-ID: <7d820835-00e6-4e15-b938-83511ba6b0e8@xen.org>
Date: Mon, 25 Mar 2024 09:28:00 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xencontrol: Fix return values and simplify failure paths
To: win-pv-devel@lists.xenproject.org
References: <20240315101133.1286-1-omeg@invisiblethingslab.com>
Content-Language: en-US
Reply-To: paul@xen.org
Organization: Xen Project
In-Reply-To: <20240315101133.1286-1-omeg@invisiblethingslab.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 15/03/2024 10:11, Rafał Wojdyła wrote:
> XcGnttabRevokeForeignAccess() and XcGnttabUnmapForeignPages()
> could return error values even when the calls succeeded. This was because
> GetLastError() was used to get the return value even if the previous call
> to DeviceIoControl() succeeded.
> 
> This commit also reworks all xencontrol functions to use simpler control
> paths in failure cases.
> 
> Signed-off-by: Rafał Wojdyła <omeg@invisiblethingslab.com>
> ---
>   src/xencontrol/xencontrol.c | 268 ++++++++++++++++--------------------
>   1 file changed, 116 insertions(+), 152 deletions(-)
> 

Acked-by: Paul Durrant <paul@xen.org>



From win-pv-devel-bounces@lists.xenproject.org Mon Mar 25 10:59:41 2024
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Mon, 25 Mar 2024 10:59:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.697699.1088739 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1roi3F-0005ST-IF; Mon, 25 Mar 2024 10:59:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 697699.1088739; Mon, 25 Mar 2024 10:59:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1roi3F-0005SM-Fn; Mon, 25 Mar 2024 10:59:37 +0000
Received: by outflank-mailman (input) for mailman id 697699;
 Mon, 25 Mar 2024 10:59:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JU+D=K7=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1roi3D-0005SG-NC
 for win-pv-devel@lists.xenproject.org; Mon, 25 Mar 2024 10:59:35 +0000
Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com
 [2a00:1450:4864:20::432])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c337e0bb-ea96-11ee-a1ef-f123f15fe8a2;
 Mon, 25 Mar 2024 11:59:33 +0100 (CET)
Received: by mail-wr1-x432.google.com with SMTP id
 ffacd0b85a97d-34175878e3cso3000575f8f.0
 for <win-pv-devel@lists.xenproject.org>; Mon, 25 Mar 2024 03:59:33 -0700 (PDT)
Received: from [192.168.21.70] (54-240-197-225.amazon.com. [54.240.197.225])
 by smtp.gmail.com with ESMTPSA id
 r25-20020adfa159000000b00341b69e9162sm8605777wrr.90.2024.03.25.03.59.32
 for <win-pv-devel@lists.xenproject.org>
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 25 Mar 2024 03:59:32 -0700 (PDT)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c337e0bb-ea96-11ee-a1ef-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1711364373; x=1711969173; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:organization:reply-to
         :content-language:references:to:subject:user-agent:mime-version:date
         :message-id:from:from:to:cc:subject:date:message-id:reply-to;
        bh=m6ZU4I+4VhnW/yZregQSEopiBDqlFbkuVZCY3uR1Iy0=;
        b=FFOzXj0gSSI2xKiP5U/Ea9bxTR7bKJEjddAdAAgvtNUl8YZUqNYcHcaT7yAzs2crKl
         kkiY0ZnQnkLwASsN850OMkozFo5QQI6oRVvdbMyrYZXMXBlheKsh/EH4wxzZ79hZNzIA
         nN2jx9X4OqI0wmnNIqJ38ZDds/mE/FCpOHEbkzQrrWJ3LP88kaZlMQ3QBSapLqeSkpdq
         zVFrn76aCUaPiUmMtaNVOGkBRzqP2/h3gVNRREUjxi6Hegn+pTAxQCyrMUuIrtOwLmxq
         NGQearwiZCaa+uah8dVECoEX/MguRaUSLCdmWzq5I0ExMR2HuhBde1IJr6T1wcJ/RDiB
         +lJA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1711364373; x=1711969173;
        h=content-transfer-encoding:in-reply-to:organization:reply-to
         :content-language:references:to:subject:user-agent:mime-version:date
         :message-id:from:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=m6ZU4I+4VhnW/yZregQSEopiBDqlFbkuVZCY3uR1Iy0=;
        b=QBWlGkFfdj+dTuwNWLzxugMTfpO4UKIf6A+14UVYrIFo9o0JzbHeCry4/5qg0JjC05
         OrOe3pq0Te+bSMbFC5qnyQbkxwq8f0hHNtiPvs7kW7jfkRa1HS+2I9A9r/U/z3GxyNFH
         0lS+qzZ3HwXZ/p5uTuk87hzK2bewSUNkoGxZ8LGqMx4cQWF9h+Lu9+QgtJ+R1kspFa34
         oBNxo3e+0AlksDERX9Dk/W/LUJ7zL6y9jmSBmA+xzzOwxpbQC3rRFV0zC3CBGIh1Dq74
         WWY7qUA9wWXglj781AWnXYq/7PQaUA6afTKueaCcUkmtocIJCE79nafem3LnhLItS6HL
         q41Q==
X-Gm-Message-State: AOJu0Yx6K3J4gDdZRran1BfdBWSx7eTqa8l0l6k2+jmqNruomlcR8FqQ
	SxiGwAbjPucP/4FD7wF0iUysaNnncMAIDi6eyfveyrl2ZqbK2xoPvWlxvYxyYi4=
X-Google-Smtp-Source: AGHT+IElHQs14i4Q1s95nlxwE94zSf5uLpkyRaLqPptSHz7f76K0xlp8riu+Ki8U2NrkYwf1r5/J7g==
X-Received: by 2002:a05:6000:c8a:b0:33e:7333:d459 with SMTP id dp10-20020a0560000c8a00b0033e7333d459mr4286550wrb.49.1711364372740;
        Mon, 25 Mar 2024 03:59:32 -0700 (PDT)
From: Paul Durrant <xadimgnik@gmail.com>
X-Google-Original-From: Paul Durrant <paul@xen.org>
Message-ID: <ee198ffb-9407-4bd4-81b2-8a11c1d83976@xen.org>
Date: Mon, 25 Mar 2024 10:59:31 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Asynchronous power handling.
To: win-pv-devel@lists.xenproject.org
References: <20240318152851.1558-1-owen.smith@cloud.com>
Content-Language: en-US
Reply-To: paul@xen.org
Organization: Xen Project
In-Reply-To: <20240318152851.1558-1-owen.smith@cloud.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 18/03/2024 15:28, Owen Smith wrote:
> From: Martin Harvey <Martin.Harvey@citrix.com>
> 
> Use completion routines and IO_WORKITEMs to handle power state transitions,
> allowing execution to run at PASSIVE_LEVEL without blocking the current thread,
> and avoid blocking all power IRPs while another is executing (power IRPs should
> be synchronized by the power manager, but this doesnt appear to be the case in
> all situations).
> 
> Signed-off-by: Martin Harvey <martin.harvey@citrix.com>
> 
> Refactored, wrong format comments removed
> Signed-off-by: Owen Smith <owen.smith@cloud.com>
> 
> ---
>   src/xenvif/fdo.c | 810 +++++++++++++++++++++++++++++------------------
>   src/xenvif/pdo.c | 263 +++++++--------
>   2 files changed, 602 insertions(+), 471 deletions(-)
> 

Acked-by: Paul Durrant <paul@xen.org>



