From win-pv-devel-bounces@lists.xenproject.org Thu Mar 03 15:06:53 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Thu, 03 Mar 2022 15:06:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.283425.482424 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nPn2a-00081p-5X; Thu, 03 Mar 2022 15:06:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 283425.482424; Thu, 03 Mar 2022 15:06:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nPn2a-00081h-2S; Thu, 03 Mar 2022 15:06:52 +0000
Received: by outflank-mailman (input) for mailman id 283425;
 Thu, 03 Mar 2022 15:06:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7d/n=TO=citrix.com=prvs=05467055d=owen.smith@srs-se1.protection.inumbo.net>)
 id 1nPn2Y-00081a-78
 for win-pv-devel@lists.xenproject.org; Thu, 03 Mar 2022 15:06:50 +0000
Received: from esa1.hc3370-68.iphmx.com (esa1.hc3370-68.iphmx.com
 [216.71.145.142]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8a78a9e8-9b03-11ec-8eba-a37418f5ba1a;
 Thu, 03 Mar 2022 16:06:48 +0100 (CET)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8a78a9e8-9b03-11ec-8eba-a37418f5ba1a
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1646320008;
  h=from:to:cc:subject:date:message-id:mime-version:
   content-transfer-encoding;
  bh=ODB4nyiqxdgerwnAKFyFQ6GkqVn/XHDPlhGnLMOoG/k=;
  b=Epyc5YnVzI3L7OVoRbeILUtFOrTHpQbQR5gYKe/uO4Xo4MuAuJmue1XP
   645OvzPFOdeHgS0xQKPW2cccY8q8dNZfInuV/tv/YfJ+vNnR02VEVxz7k
   N5tfgQ9ADtHbE18x8fPozDlEEG6ux5vN6ed/W99/AGqFu4z1VAxIAUJJR
   Q=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
X-SBRS: 5.1
X-MesageID: 65803105
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:LGCZYqgjfr8z0gux/6Nh5dNoX161AxAKZh0ujC45NGQN5FlHY01je
 htvX2uPMqyCYmX9e9h0OYnl8BhSvsKDzIdnGQJqrXgyEXwb9cadCdqndUqhZCn6wu8v7a5EA
 2fyTvGacajYm1eF/k/F3oDJ9CU6jefSLlbFILas1hpZHGeIcw98z0M78wIFqtQw24LhWFvW4
 YmaT/D3YzdJ5RYlagr41IrbwP9flKyaVOQw5wFWiVhj5TcyplFNZH4tDfjZw0jQG+G4KtWSV
 efbpIxVy0uCl/sb5nFJpZ6gGqECaua60QFjERO6UYD66vRJjnRaPqrWqJPwwKqY4tmEt4kZ9
 TlDiXC/YSMYGJ32s7ghaQRRLQggH49o0qDMG1Hq5KR/z2WeG5ft6/BnDUVwNowE4OdnR2pJ8
 JT0KhhUMErF3bjvhuvmFK883azPL+GyVG8bknZ/wDefFvMnXZDrSKTW/95Imjw3g6iiGN6AO
 ppBMmM3MXwsZTV9HllHCqMezNv0m3nOeiwB8GKMt6setj27IAtZj+G2bYu9lsaxbdpUmAOAu
 CfK8nr0BjkeNceD0nyV/3S0nOjNkCjnHoUIG9WFGuVC2QPJgDZJUVtPCAX98aLRZlOCt8x3D
 30yyhAF8ZQI6UWVZ4fffTOSv3W+l0tJMzZPKNES5AaIw6vSxg+WAGkYUzJMAOAbWN8KqS8Cj
 QHQwY6wbdB7mPjMEC/GqO/Ixd+nEXVNdQc/iTk4oRzpCjUJiKU6lVrxQ9lqC8ZZZfWlSGirk
 1hmQMXT7oj/bPLnNY3mpTgrYBr2//AlqzLZAC2NBApJCSsjOeaYi3SAswSz0Bq5BN/xoqO9l
 HYFgdOCy+sFEIuAkieAKM1UQu33varbbGyH3gM2d3XEy9hL0yT5FWy3yGsjTHqFz+5eIWO5C
 KMtkVk5CGBv0IuCMvYsPtPZ5zUCxqn8D9X1Ps04nfIVCqWdgDSvpXk0DWbJhjiFuBF1zckXY
 M7LGe7xXSdyIfk2l1KeGrZCuYLHMwhjnAs/s7iglE/5uVdfDVbIIYo43KymNbhovPvZ+1yOq
 76y9aKikn1ibQE3WQGPmaZ7ELzABSNT6UzewyCPStO+Hw==
IronPort-HdrOrdr: A9a23:QNGVDqGEQSYMQB9LpLqE08eALOsnbusQ8zAXPiFKOGRom6mj/f
 xG885rsSMc5AxhPU3I3OrwWpVoIkm9yXcW2/h3AV7KZmCP01dAbrsSjrcKqAeQfREWmNQtrJ
 uIsJITNDQzNzVHZArBjjVQ2uxOoeW6zA==
X-IronPort-AV: E=Sophos;i="5.90,151,1643691600"; 
   d="scan'208";a="65803105"
From: Owen Smith <owen.smith@citrix.com>
To: <win-pv-devel@lists.xenproject.org>
CC: Owen Smith <owen.smith@citrix.com>
Subject: [PATCH] Use List macros to manipulate lists
Date: Thu, 3 Mar 2022 15:06:21 +0000
Message-ID: <20220303150621.1223-1-owen.smith@citrix.com>
X-Mailer: git-send-email 2.33.0.windows.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

Signed-off-by: Owen Smith <owen.smith@citrix.com>
---
 src/xenbus/cache.c | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)

diff --git a/src/xenbus/cache.c b/src/xenbus/cache.c
index 8dcb3a7..576d6ca 100644
--- a/src/xenbus/cache.c
+++ b/src/xenbus/cache.c
@@ -688,12 +688,11 @@ CacheSpill(
     if (Cache->Count <= Count)
         goto done;
 
-    ListEntry = Cache->SlabList.Blink;
-    while (ListEntry != &Cache->SlabList) {
-        PLIST_ENTRY         Prev = ListEntry->Blink;
+    while (!IsListEmpty(&Cache->SlabList)) {
         PXENBUS_CACHE_SLAB  Slab;
 
-        ASSERT(!IsListEmpty(&Cache->SlabList));
+        ListEntry = Cache->SlabList.Blink;
+        ASSERT(ListEntry != &Cache->SlabList);
 
         Slab = CONTAINING_RECORD(ListEntry, XENBUS_CACHE_SLAB, ListEntry);
 
@@ -705,8 +704,6 @@ CacheSpill(
             break;
 
         CacheDestroySlab(Cache, Slab);
-
-        ListEntry = Prev;
     }
 
     CacheAudit(Cache);
-- 
2.33.0.windows.2



From win-pv-devel-bounces@lists.xenproject.org Mon Mar 07 07:46:36 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Mon, 07 Mar 2022 07:46:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.285688.484929 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nR84h-0003iB-0t; Mon, 07 Mar 2022 07:46:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 285688.484929; Mon, 07 Mar 2022 07:46:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nR84g-0003i3-U9; Mon, 07 Mar 2022 07:46:34 +0000
Received: by outflank-mailman (input) for mailman id 285688;
 Mon, 07 Mar 2022 07:46:34 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=f5t1=TS=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1nR84g-0003hw-0e
 for win-pv-devel@lists.xenproject.org; Mon, 07 Mar 2022 07:46:34 +0000
Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com
 [2a00:1450:4864:20::331])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b46c5a58-9dea-11ec-8eba-a37418f5ba1a;
 Mon, 07 Mar 2022 08:46:33 +0100 (CET)
Received: by mail-wm1-x331.google.com with SMTP id
 q7-20020a7bce87000000b00382255f4ca9so10267591wmj.2
 for <win-pv-devel@lists.xenproject.org>; Sun, 06 Mar 2022 23:46:31 -0800 (PST)
Received: from ?IPV6:2a00:23c5:5785:9a01:ad9a:ab78:5748:a7ec?
 ([2a00:23c5:5785:9a01:ad9a:ab78:5748:a7ec])
 by smtp.gmail.com with ESMTPSA id
 n7-20020a05600c3b8700b00389a6241669sm1611856wms.33.2022.03.06.23.46.30
 for <win-pv-devel@lists.xenproject.org>
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Sun, 06 Mar 2022 23:46:31 -0800 (PST)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b46c5a58-9dea-11ec-8eba-a37418f5ba1a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=message-id:date:mime-version:user-agent:reply-to:subject
         :content-language:to:references:from:in-reply-to
         :content-transfer-encoding;
        bh=05hDLW/y9O7RzRPJ/ATbD6Nm4zlehbqv34bDpAC9GPA=;
        b=kY0oAuWLAsIz98qeqqvtWJMK9KrRGC1/2AebpWXOZcisVJSPzS/MfEibhrT8oSwktu
         ksreL/Xy700jhCOupgLZOviGRapuOh/0vEqpaETacDeJLLD9PvTaOiYDhFp2NC8WbA8N
         osxPeGUUh4AjzR2t5Yg80OmF2RsbvoazSkI+gmRx/xcn9Q+dexv2tMhedVaGB/5nz2l4
         J7ogGtUX05/P68eALBoZSUi5cevjK2ym9dxDEYZWpo9pw+7Ee62mdwJWim9tmKCV75nu
         dIsMDMTzmM4ThMrP4VePmmVEme3fXmKjO12usZR5IOTeSJtBVDLO5Xjeo3PZYwG3U7eu
         pFeQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:message-id:date:mime-version:user-agent:reply-to
         :subject:content-language:to:references:from:in-reply-to
         :content-transfer-encoding;
        bh=05hDLW/y9O7RzRPJ/ATbD6Nm4zlehbqv34bDpAC9GPA=;
        b=16xc4mDgD423N8gfFJe4z9iX0W56HVRWZWTTvUY27F/ltDGH8akpD1zlZiWOgkN6ip
         a1ASlTvqj2Sju5l7J2sCDt+V51GGqr9sUgZsYBuYe6W5Y9C7nQTGy2uXCsYz2GliwLgg
         MjSQ0LwtY7aWD2RIe3j2CJIIMe0Xu6/St9Zy5SgC7AHWtGF3ZZWxt1WDBqwHcVBLaJJq
         DmSzzoXbv9yUM/4TJ2ZJ8yUP6ZNplWQ7K1d2Sui8V8T2EeW7v+EPK32y0VZhqjotIj0V
         SxORKzkgI7Yw6z1cHTYffH/8H+C2d4ivjjY8OT1AupmgnirP9N6b0O3Jp5j2EYcaVjJs
         ZZRA==
X-Gm-Message-State: AOAM533xYaAX/gpAvtuf+Ay9hGhBM+kSs7x2vHV7qX9nALOJLf7hvJpL
	gUpVWt/KDSr9ygBOQw/q8SjtGUqvV6Q=
X-Google-Smtp-Source: ABdhPJzMSZJFZ4O2ORzpdEQTZmixJDbWkMkR/LAR2oV1vYbzOm44othzM764f7xRKsmu5tvhSoHcYw==
X-Received: by 2002:a05:600c:216:b0:387:d3c0:8214 with SMTP id 22-20020a05600c021600b00387d3c08214mr8415032wmi.96.1646639191412;
        Sun, 06 Mar 2022 23:46:31 -0800 (PST)
Message-ID: <339ccea8-cfce-b199-e41b-0e399b3729cf@gmail.com>
Date: Mon, 7 Mar 2022 07:46:30 +0000
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.6.1
Reply-To: paul@xen.org
Subject: Re: [PATCH] All items in SYSTEM_PROCESSOR array may not be
 initialized
Content-Language: en-US
To: win-pv-devel@lists.xenproject.org
References: <20220228114701.1547-1-owen.smith@citrix.com>
From: "Durrant, Paul" <xadimgnik@gmail.com>
In-Reply-To: <20220228114701.1547-1-owen.smith@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 28/02/2022 11:47, Owen Smith wrote:
> The SYSTEM_PROCESSOR array is allocated to fit the maximum number of supported
> CPUs, but elements are only initialized when the SystemProcessorChangeCallback
> callback is called with KeProcessorAddCompleteNotify.
> Check if the SYSTEM_PROCESSOR structure is initialized before accessing any
> other members, and fail SystemProcessorVcpuId with STATUS_NOT_SUPPORTED for any
> uninitialized CPUs
> 
> Signed-off-by: Owen Smith <owen.smith@citrix.com>

Much nicer.

Acked-by: Paul Durrant <paul@xen.org>


From win-pv-devel-bounces@lists.xenproject.org Mon Mar 07 07:49:14 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Mon, 07 Mar 2022 07:49:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.285695.484934 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nR87G-0003nt-90; Mon, 07 Mar 2022 07:49:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 285695.484934; Mon, 07 Mar 2022 07:49:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nR87G-0003nj-5N; Mon, 07 Mar 2022 07:49:14 +0000
Received: by outflank-mailman (input) for mailman id 285695;
 Mon, 07 Mar 2022 07:49:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=f5t1=TS=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1nR87F-0003nd-5C
 for win-pv-devel@lists.xenproject.org; Mon, 07 Mar 2022 07:49:13 +0000
Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com
 [2a00:1450:4864:20::434])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 140cd05d-9deb-11ec-8eba-a37418f5ba1a;
 Mon, 07 Mar 2022 08:49:12 +0100 (CET)
Received: by mail-wr1-x434.google.com with SMTP id u10so20123456wra.9
 for <win-pv-devel@lists.xenproject.org>; Sun, 06 Mar 2022 23:49:12 -0800 (PST)
Received: from ?IPV6:2a00:23c5:5785:9a01:ad9a:ab78:5748:a7ec?
 ([2a00:23c5:5785:9a01:ad9a:ab78:5748:a7ec])
 by smtp.gmail.com with ESMTPSA id
 r4-20020a5d4984000000b001f97df6c85bsm316954wrq.1.2022.03.06.23.49.11
 for <win-pv-devel@lists.xenproject.org>
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Sun, 06 Mar 2022 23:49:11 -0800 (PST)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 140cd05d-9deb-11ec-8eba-a37418f5ba1a
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=message-id:date:mime-version:user-agent:reply-to:subject
         :content-language:to:references:from:in-reply-to
         :content-transfer-encoding;
        bh=CrKWLnMETnwK+B1Ye4JOOVX/jrio7EwJnt53lQLdSMk=;
        b=hMfS+Lw5ymTjGhVPetfRJ9bIqrnyKlxX9BkgMKRsjvzVt5WJ7a5rG31VSMxBmfjRpy
         LQkDiQ1tqtg4G4GI7xtsiVFO95Arn0NcW5ihs/ladVuOxv2NANZShKOmDDHSutUH0NLo
         pc4BOWNvRQCNDiST5ct5ZAnLPEJ2qeAEsQS7rvtdCSzzT6o5W0HDkCW9JVHd1ksPZSxc
         vOX0bo1asdjMcGDMVlq1i0TU5GIkPjJge0x7CvrffBsSyWySYyw7GUdrmuu5IkQ+n5ml
         Y/PvTPvLtnBvXN9Y/VuTejjCNLwZOIwtp+t5M613xybj4ArNJJf+yjxYpRFE09naclzI
         V7LQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:message-id:date:mime-version:user-agent:reply-to
         :subject:content-language:to:references:from:in-reply-to
         :content-transfer-encoding;
        bh=CrKWLnMETnwK+B1Ye4JOOVX/jrio7EwJnt53lQLdSMk=;
        b=Nm9q4MWuBY6mgw+dGVS5KL/RKv2J1NEWRDWxRTH7Akg7wwER6DR8TcgGAauYJk8hrL
         Q+WCWDgOmh4/MeUO9XKnH9SlPHtS9f4L6M8JCpRa6iP8YkaD17AzDoF3wrF+nI9CGxyQ
         Glw25IkxtDxvmzD83loOydIaZhlO3+B6I2RiZnAWCVS5joYNxKP8cxg29YUDEyu8kxNu
         b2xd1bQFuAjRcLwYdUy2sH4QVw2rO8x3qpJukwxKelpz1p/GyEj9WJuofUyuMQ8Feeqd
         L7b2QxrPlI4PlNxBRQby52NiV2nDBMecaKb3b5yQs+5jwzxaxlk9PhVSGHNtoWuFB++H
         s/XA==
X-Gm-Message-State: AOAM5332mHEqUs54UvRpcaCGyh6lCdJUPE6zqMfc3bIu1enW4XNJRtdu
	mvoMKY9TwximDHjbJbltyTe06qwV1xQ=
X-Google-Smtp-Source: ABdhPJxZTeEMWInu+aHopRWvbXd3WBIZecTUsjrulE7CziolkA+B4v2+IqboawkQGgbdPRqt/lBsLQ==
X-Received: by 2002:a05:6000:1a88:b0:1f1:d8df:89d3 with SMTP id f8-20020a0560001a8800b001f1d8df89d3mr7278115wry.321.1646639351742;
        Sun, 06 Mar 2022 23:49:11 -0800 (PST)
Message-ID: <93946779-5bee-8883-c64c-6602603d5ec4@gmail.com>
Date: Mon, 7 Mar 2022 07:49:10 +0000
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.6.1
Reply-To: paul@xen.org
Subject: Re: [PATCH] Use List macros to manipulate lists
Content-Language: en-US
To: win-pv-devel@lists.xenproject.org
References: <20220303150621.1223-1-owen.smith@citrix.com>
From: "Durrant, Paul" <xadimgnik@gmail.com>
In-Reply-To: <20220303150621.1223-1-owen.smith@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 03/03/2022 15:06, Owen Smith wrote:
> Signed-off-by: Owen Smith <owen.smith@citrix.com>
> ---
>   src/xenbus/cache.c | 9 +++------
>   1 file changed, 3 insertions(+), 6 deletions(-)
> 
> diff --git a/src/xenbus/cache.c b/src/xenbus/cache.c
> index 8dcb3a7..576d6ca 100644
> --- a/src/xenbus/cache.c
> +++ b/src/xenbus/cache.c
> @@ -688,12 +688,11 @@ CacheSpill(
>       if (Cache->Count <= Count)
>           goto done;
>   
> -    ListEntry = Cache->SlabList.Blink;
> -    while (ListEntry != &Cache->SlabList) {
> -        PLIST_ENTRY         Prev = ListEntry->Blink;
> +    while (!IsListEmpty(&Cache->SlabList)) {
>           PXENBUS_CACHE_SLAB  Slab;
>   
> -        ASSERT(!IsListEmpty(&Cache->SlabList));
> +        ListEntry = Cache->SlabList.Blink;
> +        ASSERT(ListEntry != &Cache->SlabList);

If we want to go down this road then this should really be 
'RemoveListHead()'. I'll fix up on commit.

Acked-by: Paul Durrant <paul@xen.org>

>   
>           Slab = CONTAINING_RECORD(ListEntry, XENBUS_CACHE_SLAB, ListEntry);
>   
> @@ -705,8 +704,6 @@ CacheSpill(
>               break;
>   
>           CacheDestroySlab(Cache, Slab);
> -
> -        ListEntry = Prev;
>       }
>   
>       CacheAudit(Cache);



From win-pv-devel-bounces@lists.xenproject.org Mon Mar 07 08:14:42 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Mon, 07 Mar 2022 08:14:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.285718.484970 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nR8Vt-0007Wu-ND; Mon, 07 Mar 2022 08:14:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 285718.484970; Mon, 07 Mar 2022 08:14:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nR8Vt-0007Wm-KG; Mon, 07 Mar 2022 08:14:41 +0000
Received: by outflank-mailman (input) for mailman id 285718;
 Mon, 07 Mar 2022 08:14:39 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=f5t1=TS=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1nR8Vr-0007WQ-Kb
 for win-pv-devel@lists.xenproject.org; Mon, 07 Mar 2022 08:14:39 +0000
Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com
 [2a00:1450:4864:20::429])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a1495151-9dee-11ec-8539-5f4723681683;
 Mon, 07 Mar 2022 09:14:38 +0100 (CET)
Received: by mail-wr1-x429.google.com with SMTP id n15so7736159wra.6
 for <win-pv-devel@lists.xenproject.org>; Mon, 07 Mar 2022 00:14:38 -0800 (PST)
Received: from ?IPV6:2a00:23c5:5785:9a01:ad9a:ab78:5748:a7ec?
 ([2a00:23c5:5785:9a01:ad9a:ab78:5748:a7ec])
 by smtp.gmail.com with ESMTPSA id
 c12-20020a05600c0a4c00b00381141f4967sm15611077wmq.35.2022.03.07.00.14.36
 for <win-pv-devel@lists.xenproject.org>
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 07 Mar 2022 00:14:37 -0800 (PST)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a1495151-9dee-11ec-8539-5f4723681683
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=message-id:date:mime-version:user-agent:reply-to:subject
         :content-language:from:to:references:in-reply-to
         :content-transfer-encoding;
        bh=tZh0oQMFaXx6DMcYtY2Sf8xNE+NVGx2aQuXukLtCnx8=;
        b=N1IF7tHz3T3VMrw5EOKA8jB8E2xW4olCjQx4gcZ+Vi9g3fVlPPdyuhzDewTs/3JQPW
         qFFIm2Pbfg9W4aJwxzr5G+7ikXCagFAdQpmNbEVGB34N6YnJQspui32tIdiFuk0LbV+9
         B8zWgnyQ4RFICIEizYvp2+P9oekaeJEjICQBbc+f8BIHLbtJGavrJioEU4hBIWc/nqBo
         4JGQKJ7ZXTaxAJLOng9bjEPCGFTkzuSpsVJQyAwPIBQeKb9fl7SG8unMsed2zMUv4fMY
         4Ic+bBSnzl19smUgd9O9CqZBY9qnl7RwJJ90Lq2CO+SznJGq1KBbSe9vWmKUGjNx0+tL
         X2/g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:message-id:date:mime-version:user-agent:reply-to
         :subject:content-language:from:to:references:in-reply-to
         :content-transfer-encoding;
        bh=tZh0oQMFaXx6DMcYtY2Sf8xNE+NVGx2aQuXukLtCnx8=;
        b=6jJJH1blVYNxXv3ldzzBr2GJjsJh9s6h7zr1ueIvroq4tabW8Q0MmoCjKvDYK9Jw7Q
         MJ6mQICdPFDX1A3BEYN04lib0uDAiwwG49Kryt5ITFWJz/v14BEu9Sq/DfCAkFlmlgPc
         i6p17HXKwmpJjgS/BU+EIxZ9QZkxZHTEIoEfsCjqt8OblZNF9rBX846twRy2NR69vZz4
         mkuRztiDjla3srrgtOIgERFCGwTw8buocLawSgLZFf1a7asbViYNncSd8sWusU+RGgn9
         wotSYZrOl524YAo/vzaGHVG8lB6d3XwDvxExcezQ+aGQzt6OVNlu8b0qYCLFZ5GWpBml
         EzOw==
X-Gm-Message-State: AOAM531Be9sZlt4mVopthwGlTB1NutEVHupnhSCLl0h6aGyZQm2USAst
	x5SqwDY0JvK0LST1JSOOPjlDrilqHO8=
X-Google-Smtp-Source: ABdhPJwEiUSzW27k1XpTQfm2GDIvIeQ45gOy6lkRzW3Au7FjjVNJ3sZY606b0+kmv9UrnGQtbrCX+g==
X-Received: by 2002:a05:6000:2cf:b0:1f0:47a0:a482 with SMTP id o15-20020a05600002cf00b001f047a0a482mr7355537wry.192.1646640877563;
        Mon, 07 Mar 2022 00:14:37 -0800 (PST)
Message-ID: <e2aed03a-735c-8a5c-749a-ddb243c4b91f@gmail.com>
Date: Mon, 7 Mar 2022 08:14:36 +0000
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.6.1
Reply-To: paul@xen.org
Subject: Re: [PATCH] Use List macros to manipulate lists
Content-Language: en-US
From: "Durrant, Paul" <xadimgnik@gmail.com>
To: win-pv-devel@lists.xenproject.org
References: <20220303150621.1223-1-owen.smith@citrix.com>
 <93946779-5bee-8883-c64c-6602603d5ec4@gmail.com>
In-Reply-To: <93946779-5bee-8883-c64c-6602603d5ec4@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 07/03/2022 07:49, Durrant, Paul wrote:
> On 03/03/2022 15:06, Owen Smith wrote:
>> Signed-off-by: Owen Smith <owen.smith@citrix.com>
>> ---
>>   src/xenbus/cache.c | 9 +++------
>>   1 file changed, 3 insertions(+), 6 deletions(-)
>>
>> diff --git a/src/xenbus/cache.c b/src/xenbus/cache.c
>> index 8dcb3a7..576d6ca 100644
>> --- a/src/xenbus/cache.c
>> +++ b/src/xenbus/cache.c
>> @@ -688,12 +688,11 @@ CacheSpill(
>>       if (Cache->Count <= Count)
>>           goto done;
>> -    ListEntry = Cache->SlabList.Blink;
>> -    while (ListEntry != &Cache->SlabList) {
>> -        PLIST_ENTRY         Prev = ListEntry->Blink;
>> +    while (!IsListEmpty(&Cache->SlabList)) {
>>           PXENBUS_CACHE_SLAB  Slab;
>> -        ASSERT(!IsListEmpty(&Cache->SlabList));
>> +        ListEntry = Cache->SlabList.Blink;
>> +        ASSERT(ListEntry != &Cache->SlabList);
> 
> If we want to go down this road then this should really be 
> 'RemoveListHead()'. I'll fix up on commit.

Oh, I see that stops us making a bunch of sanity checks. Not really 
worth it then.

   Paul

> 
> Acked-by: Paul Durrant <paul@xen.org>
> 
>>           Slab = CONTAINING_RECORD(ListEntry, XENBUS_CACHE_SLAB, 
>> ListEntry);
>> @@ -705,8 +704,6 @@ CacheSpill(
>>               break;
>>           CacheDestroySlab(Cache, Slab);
>> -
>> -        ListEntry = Prev;
>>       }
>>       CacheAudit(Cache);
> 



From win-pv-devel-bounces@lists.xenproject.org Mon Mar 07 08:55:04 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Mon, 07 Mar 2022 08:55:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.285734.484985 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nR98t-0003Xi-Sr; Mon, 07 Mar 2022 08:54:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 285734.484985; Mon, 07 Mar 2022 08:54:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nR98t-0003Xa-Q3; Mon, 07 Mar 2022 08:54:59 +0000
Received: by outflank-mailman (input) for mailman id 285734;
 Mon, 07 Mar 2022 08:54:58 +0000
Received: from winpvdrvbuild.xenproject.org ([162.242.160.44])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <paul.durrant@xenproject.org>) id 1nR98s-0003XU-TW
 for win-pv-devel@lists.xenproject.org; Mon, 07 Mar 2022 08:54:58 +0000
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
Date: Mon, 7 Mar 2022 08:55:00 +0000 (UTC)
From: paul.durrant@xenproject.org
To: win-pv-devel@lists.xenproject.org
Message-ID: <8113917.3.1646643300682.JavaMail.WINPVDRVBUILD$@winpvdrvbuild>
Subject: XENBUS-master - Build #230 - Successful
MIME-Version: 1.0
Content-Type: multipart/mixed; 
	boundary="----=_Part_2_18815191.1646643300479"
X-Jenkins-Job: XENBUS-master
X-Jenkins-Result: SUCCESS


------=_Part_2_18815191.1646643300479
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

The build system has completed build #230.

Check console output at https://winpvdrvbuild.xenproject.org:8080/job/XENBUS-master/230/console to view the results.
------=_Part_2_18815191.1646643300479--


From win-pv-devel-bounces@lists.xenproject.org Mon Mar 14 15:28:27 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Mon, 14 Mar 2022 15:28:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.290271.492244 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nTmcS-0004b8-Rh; Mon, 14 Mar 2022 15:28:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 290271.492244; Mon, 14 Mar 2022 15:28:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nTmcS-0004b0-Oe; Mon, 14 Mar 2022 15:28:24 +0000
Received: by outflank-mailman (input) for mailman id 290271;
 Mon, 14 Mar 2022 15:28:23 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LNJN=TZ=citrix.com=prvs=06555da81=martin.harvey@srs-se1.protection.inumbo.net>)
 id 1nTmcR-0004au-2G
 for win-pv-devel@lists.xenproject.org; Mon, 14 Mar 2022 15:28:23 +0000
Received: from esa6.hc3370-68.iphmx.com (esa6.hc3370-68.iphmx.com
 [216.71.155.175]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5f555ecd-a3ab-11ec-8eba-a37418f5ba1a;
 Mon, 14 Mar 2022 16:28:20 +0100 (CET)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5f555ecd-a3ab-11ec-8eba-a37418f5ba1a
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1647271700;
  h=from:to:cc:subject:date:message-id:mime-version:
   content-transfer-encoding;
  bh=jb9ooGm4wFqDoBXKe9BvkBWMfUw9CSE7c4r2GL19pi4=;
  b=B1GXXaf0xiukO2Z5ceq8XAsTAW3crlFqUq8Fs3ch9Q5rvK9u/0+qoYLW
   yf64Llk7EOq/6UCmlw0QawDwKLfFfnEzvUeNhlLotbKaeW5yz1f+CfqRO
   g6DxD+PIF/4YSPP6N5oC+KBIniP1km8Na9R7Zmo5QZcm9XDx3Cf+KSpBu
   E=;
Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
X-SBRS: 5.1
X-MesageID: 66113355
X-Ironport-Server: esa6.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:fF/5aa1Uh+ueKgBrovbD5YFxkn2cJEfYwER7XKvMYLTBsI5bpzEAm
 2AaC2nSaPeDNDD3KN4jbtm+px9SuJfdmoVhTwI/pC1hF35El5HIVI+TRqvS04J+DSFhoGZPt
 Zh2hgzodZhsJpPkjk7xdOCn9xGQ7InQLlbGILes1htZGEk1EE/NtTo5w7Rj2tUx3YDja++wk
 YiaT/P3aQfNNwFcagr424rbwP+4lK2v0N+wlgVWicFj5DcypVFMZH4sDfjZw0/DaptVBoaHq
 9Prl9lVyI97EyAFUbtJmp6jGqEDryW70QKm0hK6UID66vROS7BbPg/W+5PwZG8O4whlkeydx
 /1/8pyPZkRuIpfWldU6YxRYPwUlZ7Z/reqvzXiX6aR/zmXDenrohf5vEFs3LcsT/eMf7WNmr
 KJCbmpXN1ba2rzwkOnTpupE36zPKOHuNZkDu3cmzTjDE/s3aZvCX7/L9ZlT2zJYasVmQ6iHP
 pFCNGcHgBLoQTlSEVMUFZkCuMCSn0jDTBdJlXXPqv9ii4TU5FMoi+W8WDbPQfSVRMMQhljdq
 m/Y8mDRBhABKMfZ2TeD6mirhOLEgWX8Qo16KVGj3qc02hvJnDVVUUBIEwvgyRWktqKgc+AYK
 3M7uW0yl7I/xWubQ9nRe0biu0fR63bwROFsO+E97QiMzI/d7ACYGnUIQ1Z9VTA2iCMlbWd0j
 wHUxrsFERQq6eTIEizFqt94uBvoYUAowXk+iTjopOfvy/3qu8kNgx3GVb6P+4bl34SuSVkcL
 91nxRXSZon/b+ZWjc1XHnid2lpAQ6QlqSZvum3qspqNtF8RWWJcT9XABaLnxfhBNp2FaVKKo
 WIJncOThMhXU83SzX3SGr1SQ+jxjxpgDNE7qQQzd6TNChz3oyLzFWyuyGwWyLhV3jYsJmayP
 R67VfJ5755PJnq6BZKbkKrqY/nGOZPITIy/PtiNN4ImSsEoKGevoXE/DWbNjjuFuBV9zskC1
 WKzLJ/E4YAyUv88klJbho41jNcW+8zJ7TiKFM6hkEj/j+b2ibz8Ye5tDWZip9sRtMusyDg5O
 f4FXydW432ziNHDXxQ=
IronPort-HdrOrdr: A9a23:q2NNc6/Bmaoi3b2doxNuk+DWI+orL9Y04lQ7vn2YSXRuE/Bw8P
 re+8jztCWE8Qr5N0tQ+uxoVJPufZq+z+8Q3WByB8bBYOCOggLBR+sOgbcKqweQfREWndQ86U
 4PScZD4aXLfD1Hsfo=
X-IronPort-AV: E=Sophos;i="5.90,181,1643691600"; 
   d="scan'208";a="66113355"
From: Martin Harvey <Martin.Harvey@citrix.com>
To: <win-pv-devel@lists.xenproject.org>
CC: Martin Harvey <Martin.Harvey@citrix.com>
Subject: [PATCH] Windows PV drivers fail to set up RSS when vCPUs > 8
Date: Mon, 14 Mar 2022 15:27:46 +0000
Message-ID: <20220314152746.1712-1-Martin.Harvey@citrix.com>
X-Mailer: git-send-email 2.25.0.windows.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

Signed-off-by: Martin Harvey <Martin.Harvey@citrix.com>
---
 src/xenvif/receiver.c | 168 +++++++++++++++++++++++++++++++++++++-----
 1 file changed, 148 insertions(+), 20 deletions(-)

diff --git a/src/xenvif/receiver.c b/src/xenvif/receiver.c
index 10ac6f5..0c7b32a 100644
--- a/src/xenvif/receiver.c
+++ b/src/xenvif/receiver.c
@@ -106,6 +106,7 @@ typedef struct _XENVIF_RECEIVER_RING {
     PLIST_ENTRY                 PacketQueue;
     KDPC                        QueueDpc;
     ULONG                       QueueDpcs;
+    PROCESSOR_NUMBER            TargetProcessor;
     LIST_ENTRY                  PacketComplete;
     XENVIF_RECEIVER_HASH        Hash;
 } XENVIF_RECEIVER_RING, *PXENVIF_RECEIVER_RING;
@@ -2498,6 +2499,9 @@ __ReceiverRingInitialize(
 
     KeInitializeThreadedDpc(&(*Ring)->QueueDpc, ReceiverRingQueueDpc, *Ring);
 
+    status = KeGetProcessorNumberFromIndex((*Ring)->Index, &(*Ring)->TargetProcessor);
+    ASSERT(NT_SUCCESS(status));
+
     return STATUS_SUCCESS;
 
 fail7:
@@ -2550,6 +2554,45 @@ fail1:
     return status;
 }
 
+static NTSTATUS
+__ReceiverRingSetAffinity(
+    IN  PXENVIF_RECEIVER_RING   Ring,
+    IN  PPROCESSOR_NUMBER       Processor
+    )
+{
+    PXENVIF_RECEIVER            Receiver;
+    PXENVIF_FRONTEND            Frontend;
+    NTSTATUS status;
+
+    status = STATUS_INVALID_PARAMETER;
+    if ((Ring == NULL) || (Processor == NULL))
+        goto fail1;
+
+    Receiver = Ring->Receiver;
+    Frontend = Receiver->Frontend;
+
+    /* Always update ring target processor
+       Actually set affinities if frontend override not present.
+       Re-bind event-channel if already connected */
+
+    __ReceiverRingAcquireLock(Ring);
+
+    Ring->TargetProcessor = *Processor;
+
+    /* Don't rebind event channel at this point. */
+    KeSetTargetProcessorDpcEx(&Ring->PollDpc, &Ring->TargetProcessor);
+    KeSetTargetProcessorDpcEx(&Ring->QueueDpc, &Ring->TargetProcessor);
+
+    __ReceiverRingReleaseLock(Ring);
+
+    return STATUS_SUCCESS;
+
+fail1:
+    Error("fail1 (%08x)\n", status);
+
+    return status;
+}
+
 static FORCEINLINE NTSTATUS
 __ReceiverRingConnect(
     IN  PXENVIF_RECEIVER_RING   Ring
@@ -2560,7 +2603,6 @@ __ReceiverRingConnect(
     PFN_NUMBER                  Pfn;
     CHAR                        Name[MAXNAMELEN];
     ULONG                       Index;
-    PROCESSOR_NUMBER            ProcNumber;
     NTSTATUS                    status;
 
     Receiver = Ring->Receiver;
@@ -2637,16 +2679,17 @@ __ReceiverRingConnect(
     if (Ring->Channel == NULL)
         goto fail6;
 
-    status = KeGetProcessorNumberFromIndex(Ring->Index, &ProcNumber);
-    ASSERT(NT_SUCCESS(status));
+    status = XENBUS_EVTCHN(Bind,
+                            &Receiver->EvtchnInterface,
+                            Ring->Channel,
+                            Ring->TargetProcessor.Group,
+                            Ring->TargetProcessor.Number);
+    if (!NT_SUCCESS(status))
+        Warning("Cound not set initial receiver ring affinity: 0x%x\n", status);
+    /* You haven't specifically asked for an affinity yet, so just warn. */
 
-    KeSetTargetProcessorDpcEx(&Ring->PollDpc, &ProcNumber);
-
-    (VOID) XENBUS_EVTCHN(Bind,
-                         &Receiver->EvtchnInterface,
-                         Ring->Channel,
-                         ProcNumber.Group,
-                         ProcNumber.Number);
+    KeSetTargetProcessorDpcEx(&Ring->PollDpc, &Ring->TargetProcessor);
+    KeSetTargetProcessorDpcEx(&Ring->QueueDpc, &Ring->TargetProcessor);
 
     (VOID) XENBUS_EVTCHN(Unmask,
                          &Receiver->EvtchnInterface,
@@ -2665,11 +2708,6 @@ __ReceiverRingConnect(
     if (!NT_SUCCESS(status))
         goto fail7;
 
-    status = KeGetProcessorNumberFromIndex(Ring->Index, &ProcNumber);
-    ASSERT(NT_SUCCESS(status));
-
-    KeSetTargetProcessorDpcEx(&Ring->QueueDpc, &ProcNumber);
-
     return STATUS_SUCCESS;
 
 fail7:
@@ -3917,6 +3955,56 @@ fail1:
     return status;
 }
 
+static NTSTATUS
+__ReceiverSetQueueAffinities(
+    IN  PXENVIF_RECEIVER        Receiver,
+    IN  PPROCESSOR_NUMBER       QueueAffinities,
+    IN  ULONG                   Count
+    )
+{
+    PXENVIF_FRONTEND        Frontend;
+    ULONG                   Index;
+    NTSTATUS                status;
+    KIRQL                   Irql;
+
+    Frontend = Receiver->Frontend;
+
+    status = STATUS_INVALID_PARAMETER;
+
+    if (QueueAffinities == NULL)
+        goto fail1;
+
+    if (Count > FrontendGetNumQueues(Frontend))
+        goto fail2;
+
+    KeRaiseIrql(DISPATCH_LEVEL, &Irql);
+
+    for (Index = 0; Index < Count; Index++) {
+        PXENVIF_RECEIVER_RING   Ring = Receiver->Ring[Index];
+
+        status = __ReceiverRingSetAffinity(Ring, &QueueAffinities[Index]);
+        if (!NT_SUCCESS(status))
+            goto fail3;
+    }
+
+    KeLowerIrql(Irql);
+
+    return STATUS_SUCCESS;
+
+fail3:
+    KeLowerIrql(Irql);
+
+    Error("fail3\n");
+
+fail2:
+    Error("fail2\n");
+
+fail1:
+    Error("fail1 (%08x)\n", status);
+
+    return status;
+}
+
 NTSTATUS
 ReceiverUpdateHashMapping(
     IN  PXENVIF_RECEIVER    Receiver,
@@ -3926,10 +4014,15 @@ ReceiverUpdateHashMapping(
 {
     PXENVIF_FRONTEND        Frontend;
     PULONG                  QueueMapping;
+    PPROCESSOR_NUMBER       QueueAffinities;
     ULONG                   NumQueues;
+    ULONG                   QueuesDetermined;
+    ULONG                   QIndex;
     ULONG                   Index;
+    BOOLEAN                 MapEntryDone;
     NTSTATUS                status;
 
+
     Frontend = Receiver->Frontend;
 
     QueueMapping = __ReceiverAllocate(sizeof (ULONG) * Size);
@@ -3939,26 +4032,61 @@ ReceiverUpdateHashMapping(
         goto fail1;
 
     NumQueues = FrontendGetNumQueues(Frontend);
+    QueuesDetermined = 0;
+
+    QueueAffinities = __ReceiverAllocate(sizeof(PROCESSOR_NUMBER) * NumQueues);
+    if (QueueAffinities == NULL)
+        goto fail2;
 
     status = STATUS_INVALID_PARAMETER;
+    /* N^Squared-ish, but performed infrequently */
     for (Index = 0; Index < Size; Index++) {
-        QueueMapping[Index] = KeGetProcessorIndexFromNumber(&ProcessorMapping[Index]);
-
-        if (QueueMapping[Index] >= NumQueues)
-            goto fail2;
+        MapEntryDone = FALSE;
+        /* Existing queue meets affinity requirement for the mapping at this index? */
+        for (QIndex = 0; QIndex < QueuesDetermined; QIndex++) {
+            if ((QueueAffinities[QIndex].Group == ProcessorMapping[Index].Group) &&
+                (QueueAffinities[QIndex].Number == ProcessorMapping[Index].Number)) {
+                QueueMapping[Index] = QIndex;
+                MapEntryDone = TRUE;
+            }
+        }
+        if (!MapEntryDone) {
+            /* New queue "allocation", with new affinity, if possible */
+            if (QueuesDetermined < NumQueues) {
+                QIndex = QueuesDetermined;
+                QueueAffinities[QIndex] = ProcessorMapping[Index];
+                QueueMapping[Index] = QIndex;
+                QueuesDetermined ++;
+            } else {
+                goto fail3;
+            }
+        }
     }
 
     status = FrontendSetHashMapping(Frontend, QueueMapping, Size);
     if (!NT_SUCCESS(status))
-        goto fail3;
+        goto fail4;
+
+    status = __ReceiverSetQueueAffinities(Receiver, QueueAffinities, QueuesDetermined);
+    if (!NT_SUCCESS(status))
+        goto fail5;
 
     __ReceiverFree(QueueMapping);
+    __ReceiverFree(QueueAffinities);
 
     return STATUS_SUCCESS;
 
+fail5:
+    Error("fail5\n");
+
+fail4:
+    Error("fail4\n");
+
 fail3:
     Error("fail3\n");
 
+    __ReceiverFree(QueueAffinities);
+
 fail2:
     Error("fail2\n");
 
-- 
2.25.0.windows.1



From win-pv-devel-bounces@lists.xenproject.org Mon Mar 14 17:02:34 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Mon, 14 Mar 2022 17:02:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.290349.492405 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nTo5Z-0006d3-QX; Mon, 14 Mar 2022 17:02:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 290349.492405; Mon, 14 Mar 2022 17:02:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nTo5Z-0006cw-N7; Mon, 14 Mar 2022 17:02:33 +0000
Received: by outflank-mailman (input) for mailman id 290349;
 Mon, 14 Mar 2022 17:02:32 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LNJN=TZ=citrix.com=prvs=06555da81=martin.harvey@srs-se1.protection.inumbo.net>)
 id 1nTo5X-0006cq-OE
 for win-pv-devel@lists.xenproject.org; Mon, 14 Mar 2022 17:02:32 +0000
Received: from esa2.hc3370-68.iphmx.com (esa2.hc3370-68.iphmx.com
 [216.71.145.153]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 850da533-a3b8-11ec-853b-5f4723681683;
 Mon, 14 Mar 2022 18:02:26 +0100 (CET)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 850da533-a3b8-11ec-853b-5f4723681683
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1647277346;
  h=from:to:cc:subject:date:message-id:mime-version:
   content-transfer-encoding;
  bh=1Pl06lmCXJPTq+rmHUr9SB0pkjGc9empZjbKZzvcBMk=;
  b=iXcS4tyCiCem/Ir8GLLqXdrO2xrUJ4BBp0FBsci9ZC9D6qFQFlguIy84
   zFfM2koBxEsv+0uGQ2wqJHspFOviO8f+LCG2fwG5wio88XX1IBhRaAjNA
   4FHreMLFLJ62ff1QUSvmoH5sYNFP2lTkcsZOTNufZ/lZxReFsPEyG1Inb
   E=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
X-SBRS: 5.1
X-MesageID: 66207497
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:YIT+1K93jlAOTYK9uvEmDrUD0n6TJUtcMsCJ2f8bNWPcYEJGY0x3y
 mVMXzuPO63fYzbzeNsgYY609EsD65fdmtMwTANl/3o8E34SpcT7XtnIdU2Y0wF+jyHgoOCLy
 +1EN7Es+ehtFie0Si+Fa+Sn9T8mvU2xbuKU5NTsY0idfic5DnZ54f5fs7Rh2NQw2oHlW1nlV
 e7a+KUzBnf0g1aYDUpMg06zgEsHUCPa4W5wUvQWPJinjXeG/5UnJMt3yZKZdhMUdrJ8DO+iL
 9sv+Znilo/vE7XBPfv++lrzWhVirrc/pmFigFIOM0SpqkAqSiDfTs/XnRfTAKtao2zhojx/9
 DlCncaBdl8QFarmosEMTygDGg5xAop86JaSdBBTseTLp6HHW37lwvEoB0AqJ4wIvO1wBAmi9
 9RBdmpLNErawbvrnvTrEYGAhex6RCXvFIYWoG1hy3fZBOw4TIHrSKTW/95Imjw3g6iiGN6AO
 ZBBNmY+M3wsZTUfFEgHL84AoNy5jzrneDZCjF2xi7QOtj27IAtZj+G2bYu9lsaxbdpUmAOAu
 CfK8nr0BjkeNceD0nyV/3S0nOjNkCjnHoUIG9WFGuVC2QPJgDZJUVtPCAX98aLRZlOCt8x3c
 X0rohMFqoII0V23f4L0cBKHh0SUl0tJMzZPKNES5AaIw6vSxg+WAGkYUzJMAOAbWN8KqS8Cj
 QHQwY6wbdB7mPjMEC/GqO/Ixd+nEXJNRVLucxPoWufsDzPLhIgoxizCQd94eEJepo2kQGqgq
 9xmQcVXulnysSLp//jhlbwkq2j1znQscuLSzl+LNl9JFisjOOaYi3WAsDA3F8poIoeDVUWmt
 3MZgcWY5+1mJcjTyHPUEbpXRO72vq7t3NjgbblHRcVJythQ0yT7IdA4DM9WeS+FzfroiRe2O
 RSO6Gu9FbdYPWewbL8fXm5CI59C8EQULvy8Dqq8RoMXOvBZLVbblAkzNR/49z28yyAEzPBgU
 ap3hO7xVB72/4w8l2HoLwrcuJd2rh0DKZT7HsmqkU74jeLAOBZ4i94taTOzUwzw14vcyC29z
 jqVH5fiJ8l3OAEmXhTqzA==
IronPort-HdrOrdr: A9a23:jjb7p6DNP1tPY/7lHemg55DYdb4zR+YMi2TC1yhKJyC9Ffbo8/
 xG/c5rsyMc5wxwZJhNo7y90cq7MBbhHPxOkOos1N6ZNWGM0gaVxelZnOzfKlbbehEWmNQz6U
 4ZSdkdNOHN
X-IronPort-AV: E=Sophos;i="5.90,181,1643691600"; 
   d="scan'208";a="66207497"
From: Martin Harvey <Martin.Harvey@citrix.com>
To: <win-pv-devel@lists.xenproject.org>
CC: Martin Harvey <Martin.Harvey@citrix.com>, Martin Harvey
	<martin.harvey@citrix.com>
Subject: [PATCH] Improve Xenfilt power and default IRP handling.
Date: Mon, 14 Mar 2022 17:01:41 +0000
Message-ID: <20220314170141.1565-1-Martin.Harvey@citrix.com>
X-Mailer: git-send-email 2.25.0.windows.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

Signed-off-by: Martin Harvey <martin.harvey@citrix.com>
---
 src/common/util.h |   8 +
 src/xenfilt/fdo.c | 785 ++++++++++++---------------------------------
 src/xenfilt/pdo.c | 790 ++++++++++++----------------------------------
 3 files changed, 411 insertions(+), 1172 deletions(-)

diff --git a/src/common/util.h b/src/common/util.h
index 36a36dd..fb9b9b7 100644
--- a/src/common/util.h
+++ b/src/common/util.h
@@ -36,6 +36,14 @@
 
 #include "assert.h"
 
+typedef enum _XEN_IRP_STRATEGY {
+    XEN_IRP_PASSDOWN_TRANSPARENT,
+    XEN_IRP_HANDLE_INDISPATCH_AND_COMPLETE,
+    XEN_IRP_HANDLE_INDISPATCH_PASSDOWN,
+    XEN_IRP_HANDLE_INCOMPLETION,
+    XEN_IRP_STRATEGY_MAX
+} XEN_IRP_STRATEGY, *PXEN_IRP_STRATEGY;
+
 #define	P2ROUNDUP(_x, _a)   \
         (-(-(_x) & -(_a)))
 
diff --git a/src/xenfilt/fdo.c b/src/xenfilt/fdo.c
index 63fa7b3..5d43d01 100644
--- a/src/xenfilt/fdo.c
+++ b/src/xenfilt/fdo.c
@@ -59,11 +59,6 @@ struct _XENFILT_FDO {
     PDEVICE_OBJECT                  PhysicalDeviceObject;
     CHAR                            Name[MAXNAMELEN];
 
-    PXENFILT_THREAD                 SystemPowerThread;
-    PIRP                            SystemPowerIrp;
-    PXENFILT_THREAD                 DevicePowerThread;
-    PIRP                            DevicePowerIrp;
-
     MUTEX                           Mutex;
     LIST_ENTRY                      List;
     ULONG                           References;
@@ -1004,6 +999,26 @@ fail1:
     return status;
 }
 
+static NTSTATUS
+FdoDispatchDefault(
+    IN  PXENFILT_FDO    Fdo,
+    IN  PIRP            Irp
+    )
+{
+    NTSTATUS status;
+    IoSkipCurrentIrpStackLocation(Irp);
+    status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
+    if (!NT_SUCCESS(status))
+        goto fail1;
+
+    return status;
+
+fail1:
+    Error("fail1 (%08x)\n", status);
+    return status;
+}
+
+
 static NTSTATUS
 FdoDispatchPnp(
     IN  PXENFILT_FDO    Fdo,
@@ -1055,524 +1070,135 @@ FdoDispatchPnp(
         break;
 
     default:
-        status = IoAcquireRemoveLock(&Fdo->Dx->RemoveLock, Irp);
-        if (!NT_SUCCESS(status))
-            goto fail1;
-
-        IoSkipCurrentIrpStackLocation(Irp);
-
-        status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
-        IoReleaseRemoveLock(&Fdo->Dx->RemoveLock, Irp);
+        status = FdoDispatchDefault(Fdo, Irp);
         break;
     }
 
-    return status;
-
-fail1:
-    Error("fail1 (%08x)\n", status);
-
-    Irp->IoStatus.Status = status;
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
-    return status;
-}
-
-static NTSTATUS
-FdoSetDevicePowerUp(
-    IN  PXENFILT_FDO    Fdo,
-    IN  PIRP            Irp
-    )
-{
-    PIO_STACK_LOCATION  StackLocation;
-    DEVICE_POWER_STATE  DeviceState;
-    POWER_STATE         PowerState;
-    NTSTATUS            status;
-
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
-
-    ASSERT3U(DeviceState, <,  __FdoGetDevicePowerState(Fdo));
-
-    status = FdoForwardIrpSynchronously(Fdo, Irp);
-    if (!NT_SUCCESS(status))
-        goto done;
-
-    Trace("%s: %s -> %s\n",
-          __FdoGetName(Fdo),
-          DevicePowerStateName(__FdoGetDevicePowerState(Fdo)),
-          DevicePowerStateName(DeviceState));
-
-    PowerState.DeviceState = DeviceState;
-    PoSetPowerState(Fdo->Dx->DeviceObject,
-                    DevicePowerState,
-                    PowerState);
-
-    __FdoSetDevicePowerState(Fdo, DeviceState);
-
-done:
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
-    return status;
-}
-
-static NTSTATUS
-FdoSetDevicePowerDown(
-    IN  PXENFILT_FDO    Fdo,
-    IN  PIRP            Irp
-    )
-{
-    PIO_STACK_LOCATION  StackLocation;
-    DEVICE_POWER_STATE  DeviceState;
-    POWER_STATE         PowerState;
-    NTSTATUS            status;
-
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
-
-    ASSERT3U(DeviceState, >,  __FdoGetDevicePowerState(Fdo));
-
-    Trace("%s: %s -> %s\n",
-          __FdoGetName(Fdo),
-          DevicePowerStateName(__FdoGetDevicePowerState(Fdo)),
-          DevicePowerStateName(DeviceState));
-
-    PowerState.DeviceState = DeviceState;
-    PoSetPowerState(Fdo->Dx->DeviceObject,
-                    DevicePowerState,
-                    PowerState);
-
-    __FdoSetDevicePowerState(Fdo, DeviceState);
-
-    status = FdoForwardIrpSynchronously(Fdo, Irp);
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
-    return status;
-}
-
-static NTSTATUS
-FdoSetDevicePower(
-    IN  PXENFILT_FDO    Fdo,
-    IN  PIRP            Irp
-    )
-{
-    PIO_STACK_LOCATION  StackLocation;
-    DEVICE_POWER_STATE  DeviceState;
-    POWER_ACTION        PowerAction;
-    NTSTATUS            status;
-
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
-    PowerAction = StackLocation->Parameters.Power.ShutdownType;
-
-    Trace("%s: ====> (%s:%s)\n",
-          __FdoGetName(Fdo),
-          DevicePowerStateName(DeviceState), 
-          PowerActionName(PowerAction));
-
-    if (DeviceState == __FdoGetDevicePowerState(Fdo)) {
-        status = FdoForwardIrpSynchronously(Fdo, Irp);
-        IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
-        goto done;
-    }
-
-    status = (DeviceState < __FdoGetDevicePowerState(Fdo)) ?
-             FdoSetDevicePowerUp(Fdo, Irp) :
-             FdoSetDevicePowerDown(Fdo, Irp);
-
-done:
-    Trace("%s: <==== (%s:%s)(%08x)\n",
-          __FdoGetName(Fdo),
-          DevicePowerStateName(DeviceState), 
-          PowerActionName(PowerAction),
-          status);
-    return status;
-}
-
-static NTSTATUS
-FdoSetSystemPowerUp(
-    IN  PXENFILT_FDO    Fdo,
-    IN  PIRP            Irp
-    )
-{
-    PIO_STACK_LOCATION  StackLocation;
-    SYSTEM_POWER_STATE  SystemState;
-    NTSTATUS            status;
-
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    SystemState = StackLocation->Parameters.Power.State.SystemState;
-
-    ASSERT3U(SystemState, <,  __FdoGetSystemPowerState(Fdo));
-
-    status = FdoForwardIrpSynchronously(Fdo, Irp);
-    if (!NT_SUCCESS(status))
-        goto done;
-
-    Trace("%s: %s -> %s\n",
-          __FdoGetName(Fdo),
-          SystemPowerStateName(__FdoGetSystemPowerState(Fdo)),
-          SystemPowerStateName(SystemState));
-
-    __FdoSetSystemPowerState(Fdo, SystemState);
-
-done:
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
-    return status;
-}
-
-static NTSTATUS
-FdoSetSystemPowerDown(
-    IN  PXENFILT_FDO     Fdo,
-    IN  PIRP            Irp
-    )
-{
-    PIO_STACK_LOCATION  StackLocation;
-    SYSTEM_POWER_STATE  SystemState;
-    NTSTATUS            status;
-
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    SystemState = StackLocation->Parameters.Power.State.SystemState;
-
-    ASSERT3U(SystemState, >,  __FdoGetSystemPowerState(Fdo));
-
-    Trace("%s: %s -> %s\n",
-          __FdoGetName(Fdo),
-          SystemPowerStateName(__FdoGetSystemPowerState(Fdo)),
-          SystemPowerStateName(SystemState));
-
-    __FdoSetSystemPowerState(Fdo, SystemState);
-
-    status = FdoForwardIrpSynchronously(Fdo, Irp);
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
-    return status;
-}
-
-static NTSTATUS
-FdoSetSystemPower(
-    IN  PXENFILT_FDO    Fdo,
-    IN  PIRP            Irp
-    )
-{
-    PIO_STACK_LOCATION  StackLocation;
-    SYSTEM_POWER_STATE  SystemState;
-    POWER_ACTION        PowerAction;
-    NTSTATUS            status;
-
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    SystemState = StackLocation->Parameters.Power.State.SystemState;
-    PowerAction = StackLocation->Parameters.Power.ShutdownType;
-
-    Trace("%s: ====> (%s:%s)\n",
-          __FdoGetName(Fdo),
-          SystemPowerStateName(SystemState), 
-          PowerActionName(PowerAction));
-
-    if (SystemState == __FdoGetSystemPowerState(Fdo)) {
-        status = FdoForwardIrpSynchronously(Fdo, Irp);
-        IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
-        goto done;
-    }
-
-    status = (SystemState < __FdoGetSystemPowerState(Fdo)) ?
-             FdoSetSystemPowerUp(Fdo, Irp) :
-             FdoSetSystemPowerDown(Fdo, Irp);
-
-done:
-    Trace("%s: <==== (%s:%s)(%08x)\n",
-          __FdoGetName(Fdo),
-          SystemPowerStateName(SystemState), 
-          PowerActionName(PowerAction),
-          status);
-    return status;
-}
-
-static NTSTATUS
-FdoQueryDevicePowerUp(
-    IN  PXENFILT_FDO    Fdo,
-    IN  PIRP            Irp
-    )
-{
-    PIO_STACK_LOCATION  StackLocation;
-    DEVICE_POWER_STATE  DeviceState;
-    NTSTATUS            status;
-
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
-
-    ASSERT3U(DeviceState, <,  __FdoGetDevicePowerState(Fdo));
-
-    status = FdoForwardIrpSynchronously(Fdo, Irp);
-
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
     return status;
 }
 
 static NTSTATUS
-FdoQueryDevicePowerDown(
+FdoDispatchPowerImmediate(
     IN  PXENFILT_FDO    Fdo,
     IN  PIRP            Irp
     )
 {
-    PIO_STACK_LOCATION  StackLocation;
-    DEVICE_POWER_STATE  DeviceState;
-    NTSTATUS            status;
-
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
+    NTSTATUS status = STATUS_INTERNAL_ERROR;
+    PIO_STACK_LOCATION StackLocation = IoGetCurrentIrpStackLocation(Irp);
+    UCHAR MinorFunction = StackLocation->MinorFunction;
 
-    ASSERT3U(DeviceState, >,  __FdoGetDevicePowerState(Fdo));
-
-    status = FdoForwardIrpSynchronously(Fdo, Irp);
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
-    return status;
-}
-
-static NTSTATUS
-FdoQueryDevicePower(
-    IN  PXENFILT_FDO    Fdo,
-    IN  PIRP            Irp
-    )
-{
-    PIO_STACK_LOCATION  StackLocation;
-    DEVICE_POWER_STATE  DeviceState;
-    POWER_ACTION        PowerAction;
-    NTSTATUS            status;
+    if (MinorFunction != IRP_MN_SET_POWER)
+        goto fail1;
 
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
-    PowerAction = StackLocation->Parameters.Power.ShutdownType;
+    POWER_STATE_TYPE PowerType = StackLocation->Parameters.Power.Type;
 
-    Trace("%s: ====> (%s:%s)\n",
-          __FdoGetName(Fdo),
-          DevicePowerStateName(DeviceState), 
-          PowerActionName(PowerAction));
+    switch (PowerType) {
+    case DevicePowerState:
+        POWER_STATE PowerState;
+        DEVICE_POWER_STATE DeviceState = StackLocation->Parameters.Power.State.DeviceState;
+        if (DeviceState >= __FdoGetDevicePowerState(Fdo))
+            goto fail3;
 
-    if (DeviceState == __FdoGetDevicePowerState(Fdo)) {
-        status = FdoForwardIrpSynchronously(Fdo, Irp);
-        IoCompleteRequest(Irp, IO_NO_INCREMENT);
+        __FdoSetDevicePowerState(Fdo, DeviceState);
+        PowerState.DeviceState = DeviceState;
+        PoSetPowerState(__FdoGetDeviceObject(Fdo),
+                        DevicePowerState,
+                        PowerState);
+    break;
+    case SystemPowerState:
+        SYSTEM_POWER_STATE SystemState = StackLocation->Parameters.Power.State.SystemState;
+        if (SystemState >= __FdoGetSystemPowerState(Fdo))
+            goto fail3;
 
-        goto done;
+        __FdoSetSystemPowerState(Fdo, SystemState);
+    break;
+    default:
+        goto fail2;
     }
 
-    status = (DeviceState < __FdoGetDevicePowerState(Fdo)) ?
-             FdoQueryDevicePowerUp(Fdo, Irp) :
-             FdoQueryDevicePowerDown(Fdo, Irp);
-
-done:
-    Trace("%s: <==== (%s:%s)(%08x)\n",
-          __FdoGetName(Fdo),
-          DevicePowerStateName(DeviceState), 
-          PowerActionName(PowerAction),
-          status);
-    return status;
-}
-
-static NTSTATUS
-FdoQuerySystemPowerUp(
-    IN  PXENFILT_FDO     Fdo,
-    IN  PIRP            Irp
-    )
-{
-    PIO_STACK_LOCATION  StackLocation;
-    SYSTEM_POWER_STATE  SystemState;
-    NTSTATUS            status;
-
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    SystemState = StackLocation->Parameters.Power.State.SystemState;
-
-    ASSERT3U(SystemState, <,  __FdoGetSystemPowerState(Fdo));
-
-    status = FdoForwardIrpSynchronously(Fdo, Irp);
-
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
+    return STATUS_SUCCESS;
 
+fail3:
+    Error("fail3 (%08x)\n", status);
+fail2:
+    Error("fail2 (%08x)\n", status);
+fail1:
+    Error("fail1 (%08x)\n", status);
     return status;
 }
 
-static NTSTATUS
-FdoQuerySystemPowerDown(
+static VOID
+FdoCompletionPower(
     IN  PXENFILT_FDO    Fdo,
     IN  PIRP            Irp
-    )
+)
 {
-    PIO_STACK_LOCATION  StackLocation;
-    SYSTEM_POWER_STATE  SystemState;
-    NTSTATUS            status;
+    PIO_STACK_LOCATION StackLocation = IoGetCurrentIrpStackLocation(Irp);
+    UCHAR MinorFunction = StackLocation->MinorFunction;
 
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    SystemState = StackLocation->Parameters.Power.State.SystemState;
-
-    ASSERT3U(SystemState, >,  __FdoGetSystemPowerState(Fdo));
-
-    status = FdoForwardIrpSynchronously(Fdo, Irp);
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
-    return status;
-}
+    if (MinorFunction != IRP_MN_SET_POWER)
+        goto fail1;
 
-static NTSTATUS
-FdoQuerySystemPower(
-    IN  PXENFILT_FDO    Fdo,
-    IN  PIRP            Irp
-    )
-{
-    PIO_STACK_LOCATION  StackLocation;
-    SYSTEM_POWER_STATE  SystemState;
-    POWER_ACTION        PowerAction;
-    NTSTATUS            status;
+    POWER_STATE_TYPE PowerType = StackLocation->Parameters.Power.Type;
 
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    SystemState = StackLocation->Parameters.Power.State.SystemState;
-    PowerAction = StackLocation->Parameters.Power.ShutdownType;
+    switch (PowerType) {
+    case DevicePowerState:
+        POWER_STATE PowerState;
+        DEVICE_POWER_STATE DeviceState = StackLocation->Parameters.Power.State.DeviceState;
+        if (DeviceState <= __FdoGetDevicePowerState(Fdo))
+            goto fail3;
 
-    Trace("%s: ====> (%s:%s)\n",
-          __FdoGetName(Fdo),
-          SystemPowerStateName(SystemState), 
-          PowerActionName(PowerAction));
+        PowerState.DeviceState = DeviceState;
+        PoSetPowerState(__FdoGetDeviceObject(Fdo),
+                        DevicePowerState,
+                        PowerState);
+        __FdoSetDevicePowerState(Fdo, DeviceState);
 
-    if (SystemState == __FdoGetSystemPowerState(Fdo)) {
-        status = FdoForwardIrpSynchronously(Fdo, Irp);
-        IoCompleteRequest(Irp, IO_NO_INCREMENT);
+    break;
+    case SystemPowerState:
+        SYSTEM_POWER_STATE SystemState = StackLocation->Parameters.Power.State.SystemState;
+        if (SystemState <= __FdoGetSystemPowerState(Fdo))
+            goto fail3;
 
-        goto done;
+        __FdoSetSystemPowerState(Fdo, SystemState);
+    break;
+    default:
+        goto fail2;
     }
 
-    status = (SystemState < __FdoGetSystemPowerState(Fdo)) ?
-             FdoQuerySystemPowerUp(Fdo, Irp) :
-             FdoQuerySystemPowerDown(Fdo, Irp);
-
-done:
-    Trace("%s: <==== (%s:%s)(%08x)\n",
-          __FdoGetName(Fdo),
-          SystemPowerStateName(SystemState), 
-          PowerActionName(PowerAction),
-          status);
+    return;
 
-    return status;
+fail3:
+    Error("fail3\n");
+fail2:
+    Error("fail2\n");
+fail1:
+    Error("fail1");
 }
 
+__drv_functionClass(IO_COMPLETION_ROUTINE)
+__drv_sameIRQL
 static NTSTATUS
-FdoDevicePower(
-    IN  PXENFILT_THREAD Self,
+FdoDispatchPowerCompletionV2(
+    IN  PDEVICE_OBJECT  DeviceObject,
+    IN  PIRP            Irp,
     IN  PVOID           Context
     )
 {
     PXENFILT_FDO        Fdo = Context;
-    PKEVENT             Event;
-
-    Event = ThreadGetEvent(Self);
-
-    for (;;) {
-        PIRP                Irp;
-        PIO_STACK_LOCATION  StackLocation;
-        UCHAR               MinorFunction;
-
-        if (Fdo->DevicePowerIrp == NULL) {
-            (VOID) KeWaitForSingleObject(Event,
-                                         Executive,
-                                         KernelMode,
-                                         FALSE,
-                                         NULL);
-            KeClearEvent(Event);
-        }
-
-        if (ThreadIsAlerted(Self))
-            break;
-
-        Irp = Fdo->DevicePowerIrp;
-
-        if (Irp == NULL)
-            continue;
 
-        Fdo->DevicePowerIrp = NULL;
-        KeMemoryBarrier();
-
-        StackLocation = IoGetCurrentIrpStackLocation(Irp);
-        MinorFunction = StackLocation->MinorFunction;
-
-        switch (StackLocation->MinorFunction) {
-        case IRP_MN_SET_POWER:
-            (VOID) FdoSetDevicePower(Fdo, Irp);
-            break;
-
-        case IRP_MN_QUERY_POWER:
-            (VOID) FdoQueryDevicePower(Fdo, Irp);
-            break;
+    UNREFERENCED_PARAMETER(DeviceObject);
 
-        default:
-            ASSERT(FALSE);
-            break;
-        }
+    if (Irp->PendingReturned)
+        IoMarkIrpPending(Irp);
 
-        IoReleaseRemoveLock(&Fdo->Dx->RemoveLock, Irp);
-    }
+    //Assumption don't need STATUS_MORE_PROCESSING_REQUIRED.
+    FdoCompletionPower(Fdo, Irp);
 
+    IoReleaseRemoveLock(&Fdo->Dx->RemoveLock, Irp);
     return STATUS_SUCCESS;
 }
 
-static NTSTATUS
-FdoSystemPower(
-    IN  PXENFILT_THREAD Self,
-    IN  PVOID           Context
-    )
-{
-    PXENFILT_FDO        Fdo = Context;
-    PKEVENT             Event;
-
-    Event = ThreadGetEvent(Self);
-
-    for (;;) {
-        PIRP                Irp;
-        PIO_STACK_LOCATION  StackLocation;
-        UCHAR               MinorFunction;
-
-        if (Fdo->SystemPowerIrp == NULL) {
-            (VOID) KeWaitForSingleObject(Event,
-                                         Executive,
-                                         KernelMode,
-                                         FALSE,
-                                         NULL);
-            KeClearEvent(Event);
-        }
-
-        if (ThreadIsAlerted(Self))
-            break;
-
-        Irp = Fdo->SystemPowerIrp;
-
-        if (Irp == NULL)
-            continue;
-
-        Fdo->SystemPowerIrp = NULL;
-        KeMemoryBarrier();
-
-        StackLocation = IoGetCurrentIrpStackLocation(Irp);
-        MinorFunction = StackLocation->MinorFunction;
-
-        switch (StackLocation->MinorFunction) {
-        case IRP_MN_SET_POWER:
-            (VOID) FdoSetSystemPower(Fdo, Irp);
-            break;
-
-        case IRP_MN_QUERY_POWER:
-            (VOID) FdoQuerySystemPower(Fdo, Irp);
-            break;
-
-        default:
-            ASSERT(FALSE);
-            break;
-        }
-
-        IoReleaseRemoveLock(&Fdo->Dx->RemoveLock, Irp);
-    }
-
-    return STATUS_SUCCESS;
-}
 
 static NTSTATUS
 FdoDispatchPower(
@@ -1582,104 +1208,125 @@ FdoDispatchPower(
 {
     PIO_STACK_LOCATION  StackLocation;
     UCHAR               MinorFunction;
-    POWER_STATE_TYPE    PowerType;
     NTSTATUS            status;
-
-    status = IoAcquireRemoveLock(&Fdo->Dx->RemoveLock, Irp);
-    if (!NT_SUCCESS(status))
-        goto fail1;
+    XEN_IRP_STRATEGY    strategy;
+    BOOLEAN HaveRemoveLock = FALSE;
 
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
     MinorFunction = StackLocation->MinorFunction;
 
-    if (MinorFunction != IRP_MN_QUERY_POWER &&
-        MinorFunction != IRP_MN_SET_POWER) {
-        IoSkipCurrentIrpStackLocation(Irp);
-
-        status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
-        IoReleaseRemoveLock(&Fdo->Dx->RemoveLock, Irp);
-
-        goto done;
-    }
-
-    PowerType = StackLocation->Parameters.Power.Type;
-
-    Trace("%s: ====> (%02x:%s)\n",
-          __FdoGetName(Fdo),
-          MinorFunction, 
-          PowerMinorFunctionName(MinorFunction)); 
-
-    switch (PowerType) {
-    case DevicePowerState:
-        IoMarkIrpPending(Irp);
-
-        ASSERT3P(Fdo->DevicePowerIrp, ==, NULL);
-        Fdo->DevicePowerIrp = Irp;
-        KeMemoryBarrier();
-
-        ThreadWake(Fdo->DevicePowerThread);
-
-        status = STATUS_PENDING;
+    //Decide how we're going to handle this IRP.
+    switch(MinorFunction) {
+    case IRP_MN_QUERY_POWER:
+        strategy = XEN_IRP_PASSDOWN_TRANSPARENT; //Dont fail, don't need to check queries.
         break;
-
-    case SystemPowerState:
-        IoMarkIrpPending(Irp);
-
-        ASSERT3P(Fdo->SystemPowerIrp, ==, NULL);
-        Fdo->SystemPowerIrp = Irp;
-        KeMemoryBarrier();
-
-        ThreadWake(Fdo->SystemPowerThread);
-
-        status = STATUS_PENDING;
+    case IRP_MN_SET_POWER:
+        POWER_STATE_TYPE PowerType = StackLocation->Parameters.Power.Type;
+
+        switch (PowerType) {
+        case DevicePowerState:
+            DEVICE_POWER_STATE DeviceState = StackLocation->Parameters.Power.State.DeviceState;
+
+            status = IoAcquireRemoveLock(&Fdo->Dx->RemoveLock, Irp);
+            if (!NT_SUCCESS(status))
+                goto fail1;
+
+            //If fails, then pending removal or already removed,
+            //freshly initialized (or not). Consider status to return...
+            HaveRemoveLock = TRUE;
+
+            if (DeviceState == __FdoGetDevicePowerState(Fdo)) {
+                strategy = XEN_IRP_PASSDOWN_TRANSPARENT;
+            } else if (DeviceState < __FdoGetDevicePowerState(Fdo)) {
+                strategy = XEN_IRP_HANDLE_INDISPATCH_PASSDOWN;
+            } else {
+                strategy = XEN_IRP_HANDLE_INCOMPLETION;
+            }
+        case SystemPowerState:
+            SYSTEM_POWER_STATE SystemState = StackLocation->Parameters.Power.State.SystemState;
+
+            status = IoAcquireRemoveLock(&Fdo->Dx->RemoveLock, Irp);
+            if (!NT_SUCCESS(status))
+                goto fail1;
+
+            //If fails, then pending removal or already removed,
+            //freshly initialized (or not). Consider status to return...
+            HaveRemoveLock = TRUE;
+
+            if (SystemState == __FdoGetSystemPowerState(Fdo)) {
+                strategy = XEN_IRP_PASSDOWN_TRANSPARENT;
+            } else if (SystemState < __FdoGetSystemPowerState(Fdo)) {
+                strategy = XEN_IRP_HANDLE_INDISPATCH_PASSDOWN;
+            } else {
+                strategy = XEN_IRP_HANDLE_INCOMPLETION;
+            }
         break;
-
+        default:
+            strategy = XEN_IRP_PASSDOWN_TRANSPARENT;
+            break;
+        }
     default:
-        IoSkipCurrentIrpStackLocation(Irp);
-
-        status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
-        IoReleaseRemoveLock(&Fdo->Dx->RemoveLock, Irp);
+        strategy = XEN_IRP_PASSDOWN_TRANSPARENT;
         break;
     }
 
-    Trace("%s: <==== (%02x:%s) (%08x)\n",
-          __FdoGetName(Fdo),
-          MinorFunction, 
-          PowerMinorFunctionName(MinorFunction),
-          status);
-
-done:
-    return status;
-
-fail1:
-    Error("fail1 (%08x)\n", status);
+    switch(strategy) {
+        case XEN_IRP_HANDLE_INDISPATCH_PASSDOWN:
+            status = FdoDispatchPowerImmediate(Fdo, Irp);
+            if (!NT_SUCCESS(status))
+                goto fail2;
+            //Fallthru.
+        case XEN_IRP_PASSDOWN_TRANSPARENT:
+        default:
+            IoSkipCurrentIrpStackLocation(Irp);
+            status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
+            if (!NT_SUCCESS(status))
+                goto fail3;
 
-    Irp->IoStatus.Status = status;
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
+            if (HaveRemoveLock)
+                IoReleaseRemoveLock(&Fdo->Dx->RemoveLock, Irp);
+        break;
+        case XEN_IRP_HANDLE_INCOMPLETION:
+            IoMarkIrpPending(Irp);
+            IoCopyCurrentIrpStackLocationToNext(Irp);
+            IoSetCompletionRoutine(Irp,
+                                FdoDispatchPowerCompletionV2,
+                                Fdo,
+                                TRUE,
+                                TRUE,
+                                TRUE);
+            status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
+            if (!NT_SUCCESS(status))
+                goto fail4;
+            status = STATUS_PENDING; //Must return STATUS_PENDING after IoMarkIrpPending, whatever happens next.
+        break;
+    }
 
     return status;
-}
-
-static NTSTATUS
-FdoDispatchDefault(
-    IN  PXENFILT_FDO    Fdo,
-    IN  PIRP            Irp
-    )
-{
-    NTSTATUS            status;
-
-    status = IoAcquireRemoveLock(&Fdo->Dx->RemoveLock, Irp);
-    if (!NT_SUCCESS(status))
-        goto fail1;
 
-    IoSkipCurrentIrpStackLocation(Irp);
-
-    status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
-    IoReleaseRemoveLock(&Fdo->Dx->RemoveLock, Irp);
+fail4:
+    //Error from lower driver, have remove lock, and completion routine.
+    //Comp routine shd be called on error to drop remove lock.
+    Error("fail4 (%08x)\n", status);
+    status = STATUS_PENDING; //Must return STATUS_PENDING after IoMarkIrpPending, whatever happens next.
+    return status;
 
+fail3:
+    //Error from lower driver, have remove lock, called down.
+    Error("fail3 (%08x)\n", status);
+    if (HaveRemoveLock)
+        IoReleaseRemoveLock(&Fdo->Dx->RemoveLock, Irp);
     return status;
 
+fail2:
+    //Immediate error, have remove lock, not called down.
+    Error("fail2 (%08x)\n", status);
+    if (HaveRemoveLock)
+        IoReleaseRemoveLock(&Fdo->Dx->RemoveLock, Irp);
+    //Fallthru.
+
 fail1:
+    //Immediate error, no remove lock, not called down.
     Error("fail1 (%08x)\n", status);
 
     Irp->IoStatus.Status = status;
@@ -1775,14 +1422,6 @@ FdoCreate(
     Fdo->LowerDeviceObject = LowerDeviceObject;
     Fdo->Type = Type;
 
-    status = ThreadCreate(FdoSystemPower, Fdo, &Fdo->SystemPowerThread);
-    if (!NT_SUCCESS(status))
-        goto fail4;
-
-    status = ThreadCreate(FdoDevicePower, Fdo, &Fdo->DevicePowerThread);
-    if (!NT_SUCCESS(status))
-        goto fail5;
-
     status = __FdoSetDeviceID(Fdo);
     if (!NT_SUCCESS(status))
         goto fail6;
@@ -1822,20 +1461,6 @@ fail7:
 fail6:
     Error("fail6\n");
 
-    ThreadAlert(Fdo->DevicePowerThread);
-    ThreadJoin(Fdo->DevicePowerThread);
-    Fdo->DevicePowerThread = NULL;
-
-fail5:
-    Error("fail5\n");
-
-    ThreadAlert(Fdo->SystemPowerThread);
-    ThreadJoin(Fdo->SystemPowerThread);
-    Fdo->SystemPowerThread = NULL;
-
-fail4:
-    Error("fail4\n");
-
     Fdo->Type = XENFILT_EMULATED_OBJECT_TYPE_UNKNOWN;
     Fdo->PhysicalDeviceObject = NULL;
     Fdo->LowerDeviceObject = NULL;
@@ -1891,14 +1516,6 @@ FdoDestroy(
     __FdoClearInstanceID(Fdo);
     __FdoClearDeviceID(Fdo);
 
-    ThreadAlert(Fdo->DevicePowerThread);
-    ThreadJoin(Fdo->DevicePowerThread);
-    Fdo->DevicePowerThread = NULL;
-
-    ThreadAlert(Fdo->SystemPowerThread);
-    ThreadJoin(Fdo->SystemPowerThread);
-    Fdo->SystemPowerThread = NULL;
-
     Fdo->Type = XENFILT_EMULATED_OBJECT_TYPE_UNKNOWN;
     Fdo->LowerDeviceObject = NULL;
     Fdo->PhysicalDeviceObject = NULL;
diff --git a/src/xenfilt/pdo.c b/src/xenfilt/pdo.c
index 741c2f3..ee2a049 100644
--- a/src/xenfilt/pdo.c
+++ b/src/xenfilt/pdo.c
@@ -56,11 +56,6 @@ struct _XENFILT_PDO {
     PDEVICE_OBJECT                  PhysicalDeviceObject;
     CHAR                            Name[MAXNAMELEN];
 
-    PXENFILT_THREAD                 SystemPowerThread;
-    PIRP                            SystemPowerIrp;
-    PXENFILT_THREAD                 DevicePowerThread;
-    PIRP                            DevicePowerIrp;
-
     PXENFILT_FDO                    Fdo;
     BOOLEAN                         Missing;
     const CHAR                      *Reason;
@@ -1101,6 +1096,26 @@ PdoEject(
     return status;
 }
 
+static NTSTATUS
+PdoDispatchDefault(
+    IN  PXENFILT_PDO    Pdo,
+    IN  PIRP            Irp
+    )
+{
+    NTSTATUS status;
+    IoSkipCurrentIrpStackLocation(Irp);
+    status = IoCallDriver(Pdo->LowerDeviceObject, Irp);
+    if (!NT_SUCCESS(status))
+        goto fail1;
+
+    return status;
+
+fail1:
+    Error("fail1 (%08x)\n", status);
+    return status;
+}
+
+
 static NTSTATUS
 PdoDispatchPnp(
     IN  PXENFILT_PDO    Pdo,
@@ -1164,528 +1179,136 @@ PdoDispatchPnp(
         break;
 
     default:
-        status = IoAcquireRemoveLock(&Pdo->Dx->RemoveLock, Irp);
-        if (!NT_SUCCESS(status))
-            goto fail1;
-
-        IoSkipCurrentIrpStackLocation(Irp);
-
-        status = IoCallDriver(Pdo->LowerDeviceObject, Irp);
-        IoReleaseRemoveLock(&Pdo->Dx->RemoveLock, Irp);
+        status = PdoDispatchDefault(Pdo, Irp);
         break;
     }
 
-    return status;
-
-fail1:
-    Error("fail1 (%08x)\n", status);
-
-    Irp->IoStatus.Status = status;
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
-    return status;
-}
-
-static NTSTATUS
-PdoSetDevicePowerUp(
-    IN  PXENFILT_PDO    Pdo,
-    IN  PIRP            Irp
-    )
-{
-    PIO_STACK_LOCATION  StackLocation;
-    DEVICE_POWER_STATE  DeviceState;
-    POWER_STATE         PowerState;
-    NTSTATUS            status;
-
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
-
-    ASSERT3U(DeviceState, <,  __PdoGetDevicePowerState(Pdo));
-
-    status = PdoForwardIrpSynchronously(Pdo, Irp);
-    if (!NT_SUCCESS(status))
-        goto done;
-
-    Trace("%s: %s -> %s\n",
-          __PdoGetName(Pdo),
-          DevicePowerStateName(__PdoGetDevicePowerState(Pdo)),
-          DevicePowerStateName(DeviceState));
-
-    PowerState.DeviceState = DeviceState;
-    PoSetPowerState(__PdoGetDeviceObject(Pdo),
-                    DevicePowerState,
-                    PowerState);
-
-    __PdoSetDevicePowerState(Pdo, DeviceState);
-
-done:
-    Irp->IoStatus.Status = status;
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
     return status;
 }
 
-static NTSTATUS
-PdoSetDevicePowerDown(
-    IN  PXENFILT_PDO    Pdo,
-    IN  PIRP            Irp
-    )
-{
-    PIO_STACK_LOCATION  StackLocation;
-    DEVICE_POWER_STATE  DeviceState;
-    POWER_STATE         PowerState;
-    NTSTATUS            status;
-
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
-
-    ASSERT3U(DeviceState, >,  __PdoGetDevicePowerState(Pdo));
-
-    Trace("%s: %s -> %s\n",
-          __PdoGetName(Pdo),
-          DevicePowerStateName(__PdoGetDevicePowerState(Pdo)),
-          DevicePowerStateName(DeviceState));
-
-    PowerState.DeviceState = DeviceState;
-    PoSetPowerState(__PdoGetDeviceObject(Pdo),
-                    DevicePowerState,
-                    PowerState);
-
-    __PdoSetDevicePowerState(Pdo, DeviceState);
-
-    status = PdoForwardIrpSynchronously(Pdo, Irp);
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
-    return status;
-}
 
 static NTSTATUS
-PdoSetDevicePower(
+PdoDispatchPowerImmediate(
     IN  PXENFILT_PDO    Pdo,
     IN  PIRP            Irp
     )
 {
-    PIO_STACK_LOCATION  StackLocation;
-    DEVICE_POWER_STATE  DeviceState;
-    POWER_ACTION        PowerAction;
-    NTSTATUS            status;
+    NTSTATUS status = STATUS_INTERNAL_ERROR;
+    PIO_STACK_LOCATION StackLocation = IoGetCurrentIrpStackLocation(Irp);
+    UCHAR MinorFunction = StackLocation->MinorFunction;
 
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
-    PowerAction = StackLocation->Parameters.Power.ShutdownType;
-
-    Trace("%s: ====> (%s:%s)\n",
-          __PdoGetName(Pdo),
-          DevicePowerStateName(DeviceState), 
-          PowerActionName(PowerAction));
-
-    if (DeviceState == __PdoGetDevicePowerState(Pdo)) {
-        status = PdoForwardIrpSynchronously(Pdo, Irp);
-        IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
-        goto done;
-    }
-
-    status = (DeviceState < __PdoGetDevicePowerState(Pdo)) ?
-             PdoSetDevicePowerUp(Pdo, Irp) :
-             PdoSetDevicePowerDown(Pdo, Irp);
-
-done:
-    Trace("%s: <==== (%s:%s)(%08x)\n",
-          __PdoGetName(Pdo),
-          DevicePowerStateName(DeviceState), 
-          PowerActionName(PowerAction),
-          status);
-    return status;
-}
-
-static NTSTATUS
-PdoSetSystemPowerUp(
-    IN  PXENFILT_PDO    Pdo,
-    IN  PIRP            Irp
-    )
-{
-    PIO_STACK_LOCATION  StackLocation;
-    SYSTEM_POWER_STATE  SystemState;
-    NTSTATUS            status;
-
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    SystemState = StackLocation->Parameters.Power.State.SystemState;
-
-    ASSERT3U(SystemState, <,  __PdoGetSystemPowerState(Pdo));
-
-    status = PdoForwardIrpSynchronously(Pdo, Irp);
-    if (!NT_SUCCESS(status))
-        goto done;
-
-    Trace("%s: %s -> %s\n",
-          __PdoGetName(Pdo),
-          SystemPowerStateName(__PdoGetSystemPowerState(Pdo)),
-          SystemPowerStateName(SystemState));
-
-    __PdoSetSystemPowerState(Pdo, SystemState);
-
-done:
-    Irp->IoStatus.Status = status;
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
-    return status;
-}
-
-static NTSTATUS
-PdoSetSystemPowerDown(
-    IN  PXENFILT_PDO     Pdo,
-    IN  PIRP            Irp
-    )
-{
-    PIO_STACK_LOCATION  StackLocation;
-    SYSTEM_POWER_STATE  SystemState;
-    NTSTATUS            status;
-
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    SystemState = StackLocation->Parameters.Power.State.SystemState;
-
-    ASSERT3U(SystemState, >,  __PdoGetSystemPowerState(Pdo));
-
-    Trace("%s: %s -> %s\n",
-          __PdoGetName(Pdo),
-          SystemPowerStateName(__PdoGetSystemPowerState(Pdo)),
-          SystemPowerStateName(SystemState));
-
-    __PdoSetSystemPowerState(Pdo, SystemState);
-
-    status = PdoForwardIrpSynchronously(Pdo, Irp);
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
-    return status;
-}
-
-static NTSTATUS
-PdoSetSystemPower(
-    IN  PXENFILT_PDO    Pdo,
-    IN  PIRP            Irp
-    )
-{
-    PIO_STACK_LOCATION  StackLocation;
-    SYSTEM_POWER_STATE  SystemState;
-    POWER_ACTION        PowerAction;
-    NTSTATUS            status;
-
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    SystemState = StackLocation->Parameters.Power.State.SystemState;
-    PowerAction = StackLocation->Parameters.Power.ShutdownType;
-
-    Trace("%s: ====> (%s:%s)\n",
-          __PdoGetName(Pdo),
-          SystemPowerStateName(SystemState), 
-          PowerActionName(PowerAction));
-
-    if (SystemState == __PdoGetSystemPowerState(Pdo)) {
-        status = PdoForwardIrpSynchronously(Pdo, Irp);
-        IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
-        goto done;
-    }
-
-    status = (SystemState < __PdoGetSystemPowerState(Pdo)) ?
-             PdoSetSystemPowerUp(Pdo, Irp) :
-             PdoSetSystemPowerDown(Pdo, Irp);
-
-done:
-    Trace("%s: <==== (%s:%s)(%08x)\n",
-          __PdoGetName(Pdo),
-          SystemPowerStateName(SystemState), 
-          PowerActionName(PowerAction),
-          status);
-    return status;
-}
-
-static NTSTATUS
-PdoQueryDevicePowerUp(
-    IN  PXENFILT_PDO    Pdo,
-    IN  PIRP            Irp
-    )
-{
-    PIO_STACK_LOCATION  StackLocation;
-    DEVICE_POWER_STATE  DeviceState;
-    NTSTATUS            status;
-
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
-
-    ASSERT3U(DeviceState, <,  __PdoGetDevicePowerState(Pdo));
-
-    status = PdoForwardIrpSynchronously(Pdo, Irp);
-
-    Irp->IoStatus.Status = status;
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
-    return status;
-}
-
-static NTSTATUS
-PdoQueryDevicePowerDown(
-    IN  PXENFILT_PDO    Pdo,
-    IN  PIRP            Irp
-    )
-{
-    PIO_STACK_LOCATION  StackLocation;
-    DEVICE_POWER_STATE  DeviceState;
-    NTSTATUS            status;
-
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
-
-    ASSERT3U(DeviceState, >,  __PdoGetDevicePowerState(Pdo));
-
-    status = PdoForwardIrpSynchronously(Pdo, Irp);
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
-    return status;
-}
-
-static NTSTATUS
-PdoQueryDevicePower(
-    IN  PXENFILT_PDO    Pdo,
-    IN  PIRP            Irp
-    )
-{
-    PIO_STACK_LOCATION  StackLocation;
-    DEVICE_POWER_STATE  DeviceState;
-    POWER_ACTION        PowerAction;
-    NTSTATUS            status;
-
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
-    PowerAction = StackLocation->Parameters.Power.ShutdownType;
+    if (MinorFunction != IRP_MN_SET_POWER)
+        goto fail1;
 
-    Trace("%s: ====> (%s:%s)\n",
-          __PdoGetName(Pdo),
-          DevicePowerStateName(DeviceState), 
-          PowerActionName(PowerAction));
+    POWER_STATE_TYPE PowerType = StackLocation->Parameters.Power.Type;
 
-    if (DeviceState == __PdoGetDevicePowerState(Pdo)) {
-        status = PdoForwardIrpSynchronously(Pdo, Irp);
-        IoCompleteRequest(Irp, IO_NO_INCREMENT);
+    switch (PowerType) {
+    case DevicePowerState:
+        POWER_STATE PowerState;
+        DEVICE_POWER_STATE DeviceState = StackLocation->Parameters.Power.State.DeviceState;
+        if (DeviceState >= __PdoGetDevicePowerState(Pdo))
+            goto fail3;
+
+        __PdoSetDevicePowerState(Pdo, DeviceState);
+        PowerState.DeviceState = DeviceState;
+        PoSetPowerState(__PdoGetDeviceObject(Pdo),
+                        DevicePowerState,
+                        PowerState);
+    break;
+    case SystemPowerState:
+        SYSTEM_POWER_STATE SystemState = StackLocation->Parameters.Power.State.SystemState;
+        if (SystemState >= __PdoGetSystemPowerState(Pdo))
+            goto fail3;
 
-        goto done;
+        __PdoSetSystemPowerState(Pdo, SystemState);
+    break;
+    default:
+        goto fail2;
     }
 
-    status = (DeviceState < __PdoGetDevicePowerState(Pdo)) ?
-             PdoQueryDevicePowerUp(Pdo, Irp) :
-             PdoQueryDevicePowerDown(Pdo, Irp);
-
-done:
-    Trace("%s: <==== (%s:%s)(%08x)\n",
-          __PdoGetName(Pdo),
-          DevicePowerStateName(DeviceState), 
-          PowerActionName(PowerAction),
-          status);
-    return status;
-}
-
-static NTSTATUS
-PdoQuerySystemPowerUp(
-    IN  PXENFILT_PDO    Pdo,
-    IN  PIRP            Irp
-    )
-{
-    PIO_STACK_LOCATION  StackLocation;
-    SYSTEM_POWER_STATE  SystemState;
-    NTSTATUS            status;
-
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    SystemState = StackLocation->Parameters.Power.State.SystemState;
-
-    ASSERT3U(SystemState, <,  __PdoGetSystemPowerState(Pdo));
-
-    status = PdoForwardIrpSynchronously(Pdo, Irp);
-
-    Irp->IoStatus.Status = status;
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
-
-    return status;
-}
-
-static NTSTATUS
-PdoQuerySystemPowerDown(
-    IN  PXENFILT_PDO    Pdo,
-    IN  PIRP            Irp
-    )
-{
-    PIO_STACK_LOCATION  StackLocation;
-    SYSTEM_POWER_STATE  SystemState;
-    NTSTATUS            status;
-
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    SystemState = StackLocation->Parameters.Power.State.SystemState;
-
-    ASSERT3U(SystemState, >,  __PdoGetSystemPowerState(Pdo));
-
-    status = PdoForwardIrpSynchronously(Pdo, Irp);
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
+    return STATUS_SUCCESS;
 
+fail3:
+    Error("fail3 (%08x)\n", status);
+fail2:
+    Error("fail2 (%08x)\n", status);
+fail1:
+    Error("fail1 (%08x)\n", status);
     return status;
 }
 
-static NTSTATUS
-PdoQuerySystemPower(
+static VOID
+PdoCompletionPower(
     IN  PXENFILT_PDO    Pdo,
     IN  PIRP            Irp
-    )
+)
 {
-    PIO_STACK_LOCATION  StackLocation;
-    SYSTEM_POWER_STATE  SystemState;
-    POWER_ACTION        PowerAction;
-    NTSTATUS            status;
+    PIO_STACK_LOCATION StackLocation = IoGetCurrentIrpStackLocation(Irp);
+    UCHAR MinorFunction = StackLocation->MinorFunction;
 
-    StackLocation = IoGetCurrentIrpStackLocation(Irp);
-    SystemState = StackLocation->Parameters.Power.State.SystemState;
-    PowerAction = StackLocation->Parameters.Power.ShutdownType;
+    if (MinorFunction != IRP_MN_SET_POWER)
+        goto fail1;
 
-    Trace("%s: ====> (%s:%s)\n",
-          __PdoGetName(Pdo),
-          SystemPowerStateName(SystemState), 
-          PowerActionName(PowerAction));
+    POWER_STATE_TYPE PowerType = StackLocation->Parameters.Power.Type;
 
-    if (SystemState == __PdoGetSystemPowerState(Pdo)) {
-        status = PdoForwardIrpSynchronously(Pdo, Irp);
-        IoCompleteRequest(Irp, IO_NO_INCREMENT);
+    switch (PowerType) {
+    case DevicePowerState:
+        POWER_STATE PowerState;
+        DEVICE_POWER_STATE DeviceState = StackLocation->Parameters.Power.State.DeviceState;
+        if (DeviceState <= __PdoGetDevicePowerState(Pdo))
+            goto fail3;
+
+        PowerState.DeviceState = DeviceState;
+        PoSetPowerState(__PdoGetDeviceObject(Pdo),
+                        DevicePowerState,
+                        PowerState);
+        __PdoSetDevicePowerState(Pdo, DeviceState);
+
+    break;
+    case SystemPowerState:
+        SYSTEM_POWER_STATE SystemState = StackLocation->Parameters.Power.State.SystemState;
+        if (SystemState <= __PdoGetSystemPowerState(Pdo))
+            goto fail3;
 
-        goto done;
+        __PdoSetSystemPowerState(Pdo, SystemState);
+    break;
+    default:
+        goto fail2;
     }
 
-    status = (SystemState < __PdoGetSystemPowerState(Pdo)) ?
-             PdoQuerySystemPowerUp(Pdo, Irp) :
-             PdoQuerySystemPowerDown(Pdo, Irp);
+    return;
 
-done:
-    Trace("%s: <==== (%s:%s)(%08x)\n",
-          __PdoGetName(Pdo),
-          SystemPowerStateName(SystemState), 
-          PowerActionName(PowerAction),
-          status);
-
-    return status;
+fail3:
+    Error("fail3\n");
+fail2:
+    Error("fail2\n");
+fail1:
+    Error("fail1");
 }
 
+__drv_functionClass(IO_COMPLETION_ROUTINE)
+__drv_sameIRQL
 static NTSTATUS
-PdoDevicePower(
-    IN  PXENFILT_THREAD Self,
+PdoDispatchPowerCompletionV2(
+    IN  PDEVICE_OBJECT  DeviceObject,
+    IN  PIRP            Irp,
     IN  PVOID           Context
     )
 {
     PXENFILT_PDO        Pdo = Context;
-    PKEVENT             Event;
-
-    Event = ThreadGetEvent(Self);
-
-    for (;;) {
-        PIRP                Irp;
-        PIO_STACK_LOCATION  StackLocation;
-        UCHAR               MinorFunction;
-
-        if (Pdo->DevicePowerIrp == NULL) {
-            (VOID) KeWaitForSingleObject(Event,
-                                         Executive,
-                                         KernelMode,
-                                         FALSE,
-                                         NULL);
-            KeClearEvent(Event);
-        }
-
-        if (ThreadIsAlerted(Self))
-            break;
-
-        Irp = Pdo->DevicePowerIrp;
-
-        if (Irp == NULL)
-            continue;
-
-        Pdo->DevicePowerIrp = NULL;
-        KeMemoryBarrier();
-
-        StackLocation = IoGetCurrentIrpStackLocation(Irp);
-        MinorFunction = StackLocation->MinorFunction;
 
-        switch (StackLocation->MinorFunction) {
-        case IRP_MN_SET_POWER:
-            (VOID) PdoSetDevicePower(Pdo, Irp);
-            break;
-
-        case IRP_MN_QUERY_POWER:
-            (VOID) PdoQueryDevicePower(Pdo, Irp);
-            break;
+    UNREFERENCED_PARAMETER(DeviceObject);
 
-        default:
-            ASSERT(FALSE);
-            break;
-        }
+    if (Irp->PendingReturned)
+        IoMarkIrpPending(Irp);
 
-        IoReleaseRemoveLock(&Pdo->Dx->RemoveLock, Irp);
-    }
+    //Assumption don't need STATUS_MORE_PROCESSING_REQUIRED.
+    PdoCompletionPower(Pdo, Irp);
 
+    IoReleaseRemoveLock(&Pdo->Dx->RemoveLock, Irp);
     return STATUS_SUCCESS;
 }
 
-static NTSTATUS
-PdoSystemPower(
-    IN  PXENFILT_THREAD Self,
-    IN  PVOID           Context
-    )
-{
-    PXENFILT_PDO        Pdo = Context;
-    PKEVENT             Event;
-
-    Event = ThreadGetEvent(Self);
-
-    for (;;) {
-        PIRP                Irp;
-        PIO_STACK_LOCATION  StackLocation;
-        UCHAR               MinorFunction;
-
-        if (Pdo->SystemPowerIrp == NULL) {
-            (VOID) KeWaitForSingleObject(Event,
-                                         Executive,
-                                         KernelMode,
-                                         FALSE,
-                                         NULL);
-            KeClearEvent(Event);
-        }
-
-        if (ThreadIsAlerted(Self))
-            break;
-
-        Irp = Pdo->SystemPowerIrp;
-
-        if (Irp == NULL)
-            continue;
-
-        Pdo->SystemPowerIrp = NULL;
-        KeMemoryBarrier();
-
-        StackLocation = IoGetCurrentIrpStackLocation(Irp);
-        MinorFunction = StackLocation->MinorFunction;
-
-        switch (StackLocation->MinorFunction) {
-        case IRP_MN_SET_POWER:
-            (VOID) PdoSetSystemPower(Pdo, Irp);
-            break;
-
-        case IRP_MN_QUERY_POWER:
-            (VOID) PdoQuerySystemPower(Pdo, Irp);
-            break;
-
-        default:
-            ASSERT(FALSE);
-            break;
-        }
-
-        IoReleaseRemoveLock(&Pdo->Dx->RemoveLock, Irp);
-    }
-
-    return STATUS_SUCCESS;
-}
 
 static NTSTATUS
 PdoDispatchPower(
@@ -1695,104 +1318,125 @@ PdoDispatchPower(
 {
     PIO_STACK_LOCATION  StackLocation;
     UCHAR               MinorFunction;
-    POWER_STATE_TYPE    PowerType;
     NTSTATUS            status;
-
-    status = IoAcquireRemoveLock(&Pdo->Dx->RemoveLock, Irp);
-    if (!NT_SUCCESS(status))
-        goto fail1;
+    XEN_IRP_STRATEGY    strategy;
+    BOOLEAN HaveRemoveLock = FALSE;
 
     StackLocation = IoGetCurrentIrpStackLocation(Irp);
     MinorFunction = StackLocation->MinorFunction;
 
-    if (MinorFunction != IRP_MN_QUERY_POWER &&
-        MinorFunction != IRP_MN_SET_POWER) {
-        IoSkipCurrentIrpStackLocation(Irp);
-
-        status = IoCallDriver(Pdo->LowerDeviceObject, Irp);
-        IoReleaseRemoveLock(&Pdo->Dx->RemoveLock, Irp);
-
-        goto done;
-    }
-
-    PowerType = StackLocation->Parameters.Power.Type;
-
-    Trace("%s: ====> (%02x:%s)\n",
-          __PdoGetName(Pdo),
-          MinorFunction, 
-          PowerMinorFunctionName(MinorFunction)); 
-
-    switch (PowerType) {
-    case DevicePowerState:
-        IoMarkIrpPending(Irp);
-
-        ASSERT3P(Pdo->DevicePowerIrp, ==, NULL);
-        Pdo->DevicePowerIrp = Irp;
-        KeMemoryBarrier();
-
-        ThreadWake(Pdo->DevicePowerThread);
-
-        status = STATUS_PENDING;
+    //Decide how we're going to handle this IRP.
+    switch(MinorFunction) {
+    case IRP_MN_QUERY_POWER:
+        strategy = XEN_IRP_PASSDOWN_TRANSPARENT; //Dont fail, don't need to check queries.
         break;
+    case IRP_MN_SET_POWER:
+        POWER_STATE_TYPE PowerType = StackLocation->Parameters.Power.Type;
 
-    case SystemPowerState:
-        IoMarkIrpPending(Irp);
+        switch (PowerType) {
+        case DevicePowerState:
+            DEVICE_POWER_STATE DeviceState = StackLocation->Parameters.Power.State.DeviceState;
 
-        ASSERT3P(Pdo->SystemPowerIrp, ==, NULL);
-        Pdo->SystemPowerIrp = Irp;
-        KeMemoryBarrier();
-
-        ThreadWake(Pdo->SystemPowerThread);
-
-        status = STATUS_PENDING;
+            status = IoAcquireRemoveLock(&Pdo->Dx->RemoveLock, Irp);
+            if (!NT_SUCCESS(status))
+                goto fail1;
+
+            //If fails, then pending removal or already removed,
+            //freshly initialized (or not). Consider status to return...
+            HaveRemoveLock = TRUE;
+
+            if (DeviceState == __PdoGetDevicePowerState(Pdo)) {
+                strategy = XEN_IRP_PASSDOWN_TRANSPARENT;
+            } else if (DeviceState < __PdoGetDevicePowerState(Pdo)) {
+                strategy = XEN_IRP_HANDLE_INDISPATCH_PASSDOWN;
+            } else {
+                strategy = XEN_IRP_HANDLE_INCOMPLETION;
+            }
+        case SystemPowerState:
+            SYSTEM_POWER_STATE SystemState = StackLocation->Parameters.Power.State.SystemState;
+
+            status = IoAcquireRemoveLock(&Pdo->Dx->RemoveLock, Irp);
+            if (!NT_SUCCESS(status))
+                goto fail1;
+
+            //If fails, then pending removal or already removed,
+            //freshly initialized (or not). Consider status to return...
+            HaveRemoveLock = TRUE;
+
+            if (SystemState == __PdoGetSystemPowerState(Pdo)) {
+                strategy = XEN_IRP_PASSDOWN_TRANSPARENT;
+            } else if (SystemState < __PdoGetSystemPowerState(Pdo)) {
+                strategy = XEN_IRP_HANDLE_INDISPATCH_PASSDOWN;
+            } else {
+                strategy = XEN_IRP_HANDLE_INCOMPLETION;
+            }
         break;
-
+        default:
+            strategy = XEN_IRP_PASSDOWN_TRANSPARENT;
+        break;
+        }
     default:
-        IoSkipCurrentIrpStackLocation(Irp);
-
-        status = IoCallDriver(Pdo->LowerDeviceObject, Irp);
-        IoReleaseRemoveLock(&Pdo->Dx->RemoveLock, Irp);
+        strategy = XEN_IRP_PASSDOWN_TRANSPARENT;
         break;
     }
 
-    Trace("%s: <==== (%02x:%s) (%08x)\n",
-          __PdoGetName(Pdo),
-          MinorFunction, 
-          PowerMinorFunctionName(MinorFunction),
-          status);
-
-done:
-    return status;
-
-fail1:
-    Error("fail1 (%08x)\n", status);
+    switch(strategy) {
+        case XEN_IRP_HANDLE_INDISPATCH_PASSDOWN:
+            status = PdoDispatchPowerImmediate(Pdo, Irp);
+            if (!NT_SUCCESS(status))
+                goto fail2;
+            //Fallthru.
+        case XEN_IRP_PASSDOWN_TRANSPARENT:
+        default:
+            IoSkipCurrentIrpStackLocation(Irp);
+            status = IoCallDriver(Pdo->LowerDeviceObject, Irp);
+            if (!NT_SUCCESS(status))
+                goto fail3;
 
-    Irp->IoStatus.Status = status;
-    IoCompleteRequest(Irp, IO_NO_INCREMENT);
+            if (HaveRemoveLock)
+                IoReleaseRemoveLock(&Pdo->Dx->RemoveLock, Irp);
+        break;
+        case XEN_IRP_HANDLE_INCOMPLETION:
+            IoMarkIrpPending(Irp);
+            IoCopyCurrentIrpStackLocationToNext(Irp);
+            IoSetCompletionRoutine(Irp,
+                                PdoDispatchPowerCompletionV2,
+                                Pdo,
+                               TRUE,
+                               TRUE,
+                               TRUE);
+            status = IoCallDriver(Pdo->LowerDeviceObject, Irp);
+            if (!NT_SUCCESS(status))
+                goto fail4;
+            status = STATUS_PENDING; //Must return STATUS_PENDING after IoMarkIrpPending, whatever happens next.
+        break;
+    }
 
     return status;
-}
-
-static NTSTATUS
-PdoDispatchDefault(
-    IN  PXENFILT_PDO    Pdo,
-    IN  PIRP            Irp
-    )
-{
-    NTSTATUS            status;
 
-    status = IoAcquireRemoveLock(&Pdo->Dx->RemoveLock, Irp);
-    if (!NT_SUCCESS(status))
-        goto fail1;
-
-    IoSkipCurrentIrpStackLocation(Irp);
-
-    status = IoCallDriver(Pdo->LowerDeviceObject, Irp);
-    IoReleaseRemoveLock(&Pdo->Dx->RemoveLock, Irp);
+fail4:
+    //Error from lower driver, have remove lock, and completion routine.
+    //Comp routine shd be called on error to drop remove lock.
+    Error("fail4 (%08x)\n", status);
+    status = STATUS_PENDING; //Must return STATUS_PENDING after IoMarkIrpPending, whatever happens next.
+    return status;
 
+fail3:
+    //Error from lower driver, have remove lock, called down.
+    Error("fail3 (%08x)\n", status);
+    if (HaveRemoveLock)
+        IoReleaseRemoveLock(&Pdo->Dx->RemoveLock, Irp);
     return status;
 
+fail2:
+    //Immediate error, have remove lock, not called down.
+    Error("fail2 (%08x)\n", status);
+    if (HaveRemoveLock)
+        IoReleaseRemoveLock(&Pdo->Dx->RemoveLock, Irp);
+    //Fallthru.
+
 fail1:
+    //Immediate error, no remove lock, not called down.
     Error("fail1 (%08x)\n", status);
 
     Irp->IoStatus.Status = status;
@@ -1907,14 +1551,6 @@ PdoCreate(
     Pdo->LowerDeviceObject = LowerDeviceObject;
     Pdo->Type = Type;
 
-    status = ThreadCreate(PdoSystemPower, Pdo, &Pdo->SystemPowerThread);
-    if (!NT_SUCCESS(status))
-        goto fail4;
-
-    status = ThreadCreate(PdoDevicePower, Pdo, &Pdo->DevicePowerThread);
-    if (!NT_SUCCESS(status))
-        goto fail5;
-
     status = PdoSetDeviceInformation(Pdo);
     if (!NT_SUCCESS(status))
         goto fail6;
@@ -1968,20 +1604,6 @@ fail7:
 fail6:
     Error("fail6\n");
 
-    ThreadAlert(Pdo->DevicePowerThread);
-    ThreadJoin(Pdo->DevicePowerThread);
-    Pdo->DevicePowerThread = NULL;
-
-fail5:
-    Error("fail5\n");
-
-    ThreadAlert(Pdo->SystemPowerThread);
-    ThreadJoin(Pdo->SystemPowerThread);
-    Pdo->SystemPowerThread = NULL;
-
-fail4:
-    Error("fail4\n");
-
     Pdo->Type = XENFILT_EMULATED_OBJECT_TYPE_UNKNOWN;
     Pdo->PhysicalDeviceObject = NULL;
     Pdo->LowerDeviceObject = NULL;
@@ -2040,14 +1662,6 @@ PdoDestroy(
 
     PdoClearDeviceInformation(Pdo);
 
-    ThreadAlert(Pdo->DevicePowerThread);
-    ThreadJoin(Pdo->DevicePowerThread);
-    Pdo->DevicePowerThread = NULL;
-
-    ThreadAlert(Pdo->SystemPowerThread);
-    ThreadJoin(Pdo->SystemPowerThread);
-    Pdo->SystemPowerThread = NULL;
-
     Pdo->Type = XENFILT_EMULATED_OBJECT_TYPE_UNKNOWN;
     Pdo->PhysicalDeviceObject = NULL;
     Pdo->LowerDeviceObject = NULL;
-- 
2.25.0.windows.1



From win-pv-devel-bounces@lists.xenproject.org Mon Mar 14 17:08:15 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Mon, 14 Mar 2022 17:08:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.290368.492420 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nToB4-0007Wr-KJ; Mon, 14 Mar 2022 17:08:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 290368.492420; Mon, 14 Mar 2022 17:08:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nToB4-0007Wj-HE; Mon, 14 Mar 2022 17:08:14 +0000
Received: by outflank-mailman (input) for mailman id 290368;
 Mon, 14 Mar 2022 17:08:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LNJN=TZ=citrix.com=prvs=06555da81=martin.harvey@srs-se1.protection.inumbo.net>)
 id 1nToB2-0007WZ-Sf
 for win-pv-devel@lists.xenproject.org; Mon, 14 Mar 2022 17:08:12 +0000
Received: from esa3.hc3370-68.iphmx.com (esa3.hc3370-68.iphmx.com
 [216.71.145.155]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 52a4c991-a3b9-11ec-853b-5f4723681683;
 Mon, 14 Mar 2022 18:08:11 +0100 (CET)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 52a4c991-a3b9-11ec-853b-5f4723681683
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1647277691;
  h=from:to:subject:date:message-id:mime-version;
  bh=egJUrvqjLWIBTQBRXWACCIvhSdOXINRstltPZPIq8Uw=;
  b=Uc2FbMp/NZbpNOxCs5eUZt8XoUt1YdhEbWfFR6D3ZTGx/OUX+j8d3NnY
   QkdqNIQflhVBgc20lYqtmgQw6f6rYD3TiOw5WRzNwU+vB5xvRp17GDG0C
   /tRd41WRUc8+gzpU470cFcjMLWEb0und1neTq6WdduGKK5Rw6VCKJNW6i
   0=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
X-SBRS: 5.1
X-MesageID: 66198096
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:H1G44atIy2D1c9hLYO0qnHFCsufnVDxeMUV32f8akzHdYApBsoF/q
 tZmKW+CO/uOYDGgLt0gPoWw8E8A6MTUxt4wGgBorno0RS1E+JbJXdiXEBz9bniYRiHhoOOLz
 Cm8hv3odp1coqr0/0/1WlTZhSAgk/nOHNIQMcacUsxLbVYMpBwJ1FQyw4bVvqYy2YLjW1jU5
 YuoyyHiEATNNwBcYzp8B52r8HuDjNyq0N/PlgVjDRzjlAa2e0g9VPrzF4noR5fLatA88tqBb
 /TC1NmEElbxpH/BPD8HfoHTKSXmSpaKVeSHZ+E/t6KK2nCurQRquko32WZ1he66RFxlkvgoo
 Oihu6BcRi9qLoTV39oHaCNIOBx9PqBH1JTrJSSw5Jn7I03uKxMAwt1rBUAye4YZ5vx2ESdF8
 vlwxDIlN07ZwbjsmfTiF7cq1p9LwMrDZevzvlllxCvFDPBgQZnZXajbzdRZwC0xloZFGvO2i
 88xN2Q2PUSYMkMn1lE/GoM6gdehqyLDchpYlG2nvvIm52Huw1kkuFTqGIWMIYHbLSlPpW6Hp
 2SD53q8DhwEOdi3zTue7mnqluLJhTn8Wo8ZCPu/7PECvbGI7jVNUltMDwL9+KTnzB7lMz5CF
 6AK0mk+hLF1ylaQdMnWYh67rX+olTIYB9UFRoXW9zqx4qbT5g+YAE0NQThAdMEquacKeNA66
 rOat4i3XGIy6dV5XVrYr+7J9m3qZUD5OEdYPUc5oR05D84PSW3ZpjbGVZ5dHaG8lbUZ8hmgk
 mnR/EDSa1j+5PPnNplXH3ia2VpARbCTF2bZAzk7uUr/smtEiHaNPdDA1LQixa8owHylZleAp
 mMYvMOV8foDC5qA/ATUHrlSReD2va7ab2SE6bKKI3XG327xk0NPgKgKuG0uTKuXGptslcDVj
 L/75loKuc470IqCZq5reYOhY/nGPoC7fekJosv8N4IUCrAoLVfv1Hg3OSa4gjC8+GBxwPpXE
 crKLq6R4YMyVP0PIMyeHLxGj9fGB0kWmAvueHwM50/+gOTEOyLNF+tt3ZnnRrlR0Z5oaT79q
 r53H8CL1w9eQKv5ZCzW+pQUNlcEMT4wApWeliCdXrXcSua6MAnN08Ps/I4=
IronPort-HdrOrdr: A9a23:kwn7iquENJLJRpzzin+lhv877skDctV00zEX/kB9WHVpm6uj5q
 eTdZUgpHvJYVMqM03I9urtBEDtexzhHP1OgbX5X43NYOCOggLBRuxfBODZogHIKmnT8fNcyL
 clU4UWMqyUMbGit7eY3OBvKadD/OW6
X-IronPort-AV: E=Sophos;i="5.90,181,1643691600"; 
   d="scan'208,217";a="66198096"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=T7EExtC8/R1DrRlBBHwGxIpG15NRFXHsB8rWIvga72335ADLjJrLQVoA4Wsf9Smbtt38lZVM1s6nsVtyT+m+n3LfBvAR+7zOx5EX/v1cw0N9TvDU9Jjcv4LT9fvr5u+8l5kfLvHcyhwqbhIVDHEOotL25K9F8KhVIfEPTAVOsNFglE3rwMvPI2pDP1oh74mrOD84XGX8dLQ8Rv2FQw/33nE8yLNzg6yvQJJrXTKTjPire/1y6hGD/ZxF0cBuulBvjov2MSxwfccbA6aPwn3tRIz7HKRdyeev2mRpqYQZJEG5i5hmaQ+4gvax5CXzkHNYdyp4h6cSOokGLTPdN4MaVg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=wJKG0m24qlkQ1yfg8jVtDmacvQs9N3TQF03XiYOq4Fk=;
 b=JlTxJHL3VM/j+j5uziu1ZAvFwiQJN+eHmZmZDOVQah8BtuiMC13Cmk8SLqiP23puXn+ic8Ktjmr52K9TJqM1Qi5yLieD+fA1dWmkC6EgCMtuWZo+qoJdo65KSV7RPMag5ZyC7pmajhjlzG97KpIbrODrdFYUVnIdYGwbB9Z4b9+GWOBqiXuN9IIlK5G2fICQwhJUwn4/M/P7EWfc2lNiLypB2Sqfo66qbo1o3id9OB6g2xlqmjwk+9a6bwKYwGU4SY4dYgybgmajv/doMADSYwZDzvskkBwWM/SNMe7F45Py7eF0rslcdniaKsyx9ulZw6F0cX10nE2E5iLvBHVk3A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=wJKG0m24qlkQ1yfg8jVtDmacvQs9N3TQF03XiYOq4Fk=;
 b=QBg0upy1Ee5q6rTtLiVcrpsP7e+jgwRtkTTgA4aSUSCbYwrnS5+l6DE2JMEhR27i1u9RS32J+k3RVk8lGndbGMXfIiK8B+62yj/ZqjX7fnAbip3XOALAU4k0iRgR7ufdOQhX5VzceMaUV4P9FEDIb5f/KjZZnOcEe0uBVVARing=
From: Martin Harvey <martin.harvey@citrix.com>
To: "win-pv-devel@lists.xenproject.org" <win-pv-devel@lists.xenproject.org>
Subject: SDV remove lock violations, and empty completion routines.
Thread-Topic: SDV remove lock violations, and empty completion routines.
Thread-Index: Adg3xa3FuEdrWapTRnWrluI7dmoyog==
Date: Mon, 14 Mar 2022 17:07:52 +0000
Message-ID: <MW4PR03MB63473840366AB6E08E60EAE79A0F9@MW4PR03MB6347.namprd03.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: c816b0bb-e303-4f82-98df-08da05dd2ce1
x-ms-traffictypediagnostic: BN8PR03MB4961:EE_
x-microsoft-antispam-prvs: <BN8PR03MB4961777974F6900BC68BCB679A0F9@BN8PR03MB4961.namprd03.prod.outlook.com>
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: uVJQSCC/5QPfvN3YBgoIOIq8xEZaSqGKimzrKpzcCG4TZdRQd8xIuYn6FKGHBx1vitH25iNnAXaDqYA4m1YGcUdJIJbQnfsAMY71Azw9wwZbyxDrugA900uiSFY3H9zZVFPJEwY3wrrTlllsz81Bm0/huiqkHC80RY1+Yd2Ql9+ojNyHe9VpOl4bsLaCq2eNTxV1htDSx8FbQ9T9OJSQcjV+fMVuIiuHJ3mUiI5HbmQTvC3LoJHCmLEXuaZkvq6ru3ye3N06odhXKrK87fhKE7GZmB7Q7j4CZz8YL4fsd0+ondpv4+wL+CyeboCridxUc6kJ28TJVFVEIC8fv60MOnBgesu3ER8iPqi7gj6nnWATyD0vIsRIohlmndKYlje2C7PcdidDMmyrD4aIJMSRXK9XnmxeYA2J8JpWAb8ueR5xRC2xGIw0IQZ6UUHfxyQrCPa0RFWMUJHIcE/zH7uWqmGBE9sQTfh5CItKxT/DpY/hUy9RAgbrQlJOZHpS4VWVvKl7T6g1phFYUAMkBUqdJmz4vhVWLcbFZmQ0OsQZtpTjd6047V9UibBGhqgKqQ+PIFGokovy3OOrVk9qI5O523ZixOK9oN1nlhIdRHj8PKqQJB8G5UaQnQxYaQmw+iLcKbrkAWZltr95AEweUoZ8jf0csslH7eBe8zLTUahDpFBaLtR5FhHefMqN4t3sYZjgZE0DbvXABlgM1OmlZGnkGQ==
x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW4PR03MB6347.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(83380400001)(316002)(2906002)(6916009)(26005)(71200400001)(558084003)(6506007)(7696005)(9686003)(186003)(8676002)(64756008)(82960400001)(76116006)(66946007)(66446008)(66476007)(66556008)(86362001)(38100700002)(52536014)(44832011)(8936002)(33656002)(38070700005)(5660300002)(508600001)(122000001)(55016003);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?F26jmBKpLzgoWaJ6CSIeth66lUxVtau1hYHMZvl2yWiahNGjxEy6NgPFo7jg?=
 =?us-ascii?Q?LuYzG4dm/mD/21mDVrA39StJOxrKHk9O1LYmENfcjxgWNg397VDHKOxdAphx?=
 =?us-ascii?Q?axfAdaaFi7dIxn4J+fKKTt6XY9GfKN+N1OstKHXwRf8FgkoSBTJokJkBRFFX?=
 =?us-ascii?Q?9brObqO7hj9Djo7xcb+KY9jzPVmY9Pr2pIlCmEE1CqEkOake6sgtktH6wpwJ?=
 =?us-ascii?Q?gMOjFBH5YfdYsd1oeVh3hkestFci72Y/9IdjMaCYQoAxVwYU2o9mRFha6UQE?=
 =?us-ascii?Q?9goNe8vy7pRynPjBrPeWeyFbiOigFrEsHBXqCVmnVRJkkiSZjvexgDgyVK65?=
 =?us-ascii?Q?MC+oUVVVkDhMGZ7oTmItVyMDx7QOQPF/epqMoECRSBPXRVCgzCRQxMVTFx61?=
 =?us-ascii?Q?s9eZiVmqhYOTMu4g+0QWZnFthK/LbuvqYlgeM8ZXDXilF1YoK3N6J6tPk/IT?=
 =?us-ascii?Q?L4JkccWQv/gFTnEc7GzhJRQ7oS4q2VZKgUPtKJeonsQO/G2ou8Icc0YMS2Pz?=
 =?us-ascii?Q?JcesfHGwb8ptwF4vayIrfOTE2l3+koV9F7MnlboVbNwg8AclxwocaBh0A0pL?=
 =?us-ascii?Q?OYA5MVbeZ9GsA1aw3ZZGeRyCftjfH51nmJzBCMMEcwC9Mf8QMwkA54CEu8NN?=
 =?us-ascii?Q?Pxb1SHKAuIj/SFi0wW+ZcvssWDLu4WZoGUMf0uz2V/FAyopPDexr2KWmgXJy?=
 =?us-ascii?Q?FZV8SrWSPk5DyluxOzBIJu8E6Xx8SwLXfxGY7kXBJf4lFI6JzGEL6yYihEPv?=
 =?us-ascii?Q?r9d50xgZ0IZ/GjN42AM1ngneW6J4pUSWrZ1igMZj/qP5H4iYhMZRNnX2YXU7?=
 =?us-ascii?Q?8D9u8CFBnrvBO44lARDDR2gBz2MC0DY/TCBJiYuUixzgqoTSy+OZkyDSR7oK?=
 =?us-ascii?Q?EPDI0jOcybhABkwVClYhbRVx9WGvMEED4Sa7P8x9qPpAMH/sfldd/dULyJa7?=
 =?us-ascii?Q?YWY+VhUMQdEqvr/tD5YflSEYhvgIOYvdlFGfGEjN1QgGcrO4K/3jJgxjh1Iz?=
 =?us-ascii?Q?qZi1AY86F9MTAzvduVa3nNDfb9WuGmkpbiq+UQkwwZaKLW189KthWPUtz+t+?=
 =?us-ascii?Q?xIP5AIipNraVe0oQ+U/vYapWlscV90KxMGIlatPmLLZeRKMPK2MLXmo9M/Mv?=
 =?us-ascii?Q?FQzEu/RQMbtxrxeFyCgpQt5SN+GO8IPqVUcf7CT28QWkkPA4ZeFrsJ0+K6WT?=
 =?us-ascii?Q?dn/LCEBA17e2K3vA0Xz3Xq/+DUAo2sDWtaZPPuaS3OvlNaLjN0raP7y+fMBU?=
 =?us-ascii?Q?yWsjtWuw2GOPPQMa0bi6NaPzBg+4tEgRCyrzhoIx9Jt9bs1exbChbiXgP7Cy?=
 =?us-ascii?Q?HSQUfSCCtkbiprQ4x8doYpT6ekctruNT6Ol9GzBLPc2YxKr3QxtBVNUHuBxH?=
 =?us-ascii?Q?CdgwWg/px+v822ol66XFlmtqq0sMiP+roSyPBQ3u43s0cjIVSJEwB87XgYzt?=
 =?us-ascii?Q?UPOdsZB55qatz+PHECgaPtseY31tAX4jbHKj+kUVuetyPyc2Gf3KYw=3D=3D?=
Content-Type: multipart/alternative;
	boundary="_000_MW4PR03MB63473840366AB6E08E60EAE79A0F9MW4PR03MB6347namp_"
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MW4PR03MB6347.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c816b0bb-e303-4f82-98df-08da05dd2ce1
X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Mar 2022 17:07:52.4771
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: t2O8Bs8V4uzHLMc2VVb2DK/8X9KHpbavjb6t2zhmpVLspV+E22s5stk5ixroHwtSaXbVbAlwTxYwU8NtEy3GbI6FaSBw0HhWM1zanHlLPyo=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR03MB4961
X-OriginatorOrg: citrix.com

--_000_MW4PR03MB63473840366AB6E08E60EAE79A0F9MW4PR03MB6347namp_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable


Re changeset: 7817a9b7fe421654c3d1039849a175380f9e9ecb

There is a rather nicer way of refactoring this along similar lines to [PAT=
CH] Improve Xenfilt power and default IRP handling.

If Owen and I can update our upstream refs for our own patch queues, I can =
submit an additional patch.

MH.

--_000_MW4PR03MB63473840366AB6E08E60EAE79A0F9MW4PR03MB6347namp_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr=
osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:=
//www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
<meta name=3D"Generator" content=3D"Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
	{font-family:"Cambria Math";
	panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
	{font-family:DengXian;
	panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
	{font-family:Calibri;
	panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
	{font-family:"\@DengXian";
	panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0cm;
	font-size:11.0pt;
	font-family:"Calibri",sans-serif;}
span.EmailStyle17
	{mso-style-type:personal-compose;
	font-family:"Calibri",sans-serif;
	color:windowtext;}
.MsoChpDefault
	{mso-style-type:export-only;
	font-family:"Calibri",sans-serif;}
@page WordSection1
	{size:612.0pt 792.0pt;
	margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
	{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=3D"EN-GB" link=3D"#0563C1" vlink=3D"#954F72" style=3D"word-wrap:=
break-word">
<div class=3D"WordSection1">
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">Re changeset: 7817a9b7fe421654c3d1039849a175380f9e9e=
cb<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">There is a rather nicer way of refactoring this alon=
g similar lines to [PATCH] Improve Xenfilt power and default IRP handling.<=
o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">If Owen and I can update our upstream refs for our o=
wn patch queues, I can submit an additional patch.<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">MH.<o:p></o:p></p>
</div>
</body>
</html>

--_000_MW4PR03MB63473840366AB6E08E60EAE79A0F9MW4PR03MB6347namp_--


From win-pv-devel-bounces@lists.xenproject.org Tue Mar 15 11:19:10 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Tue, 15 Mar 2022 11:19:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.290745.493208 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nU5Cm-0003T9-EF; Tue, 15 Mar 2022 11:19:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 290745.493208; Tue, 15 Mar 2022 11:19:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nU5Cm-0003T1-B6; Tue, 15 Mar 2022 11:19:08 +0000
Received: by outflank-mailman (input) for mailman id 290745;
 Tue, 15 Mar 2022 11:19:06 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bd58=T2=citrix.com=prvs=066e9a2b4=martin.harvey@srs-se1.protection.inumbo.net>)
 id 1nU5Ck-0003St-1W
 for win-pv-devel@lists.xenproject.org; Tue, 15 Mar 2022 11:19:06 +0000
Received: from esa3.hc3370-68.iphmx.com (esa3.hc3370-68.iphmx.com
 [216.71.145.155]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b778c3d0-a451-11ec-8eba-a37418f5ba1a;
 Tue, 15 Mar 2022 12:19:03 +0100 (CET)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b778c3d0-a451-11ec-8eba-a37418f5ba1a
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1647343143;
  h=from:to:subject:date:message-id:references:in-reply-to:
   content-transfer-encoding:mime-version;
  bh=eETD53zRvhBiQbO7cdvrfJ1RCF19NWp7jHV1MxrkUs0=;
  b=HIDiP5CQKXHkuK4edg/MYL3ApfkX5bEex5M+OI8RqYLDDNoNb4zECrD1
   meMNxjYRhObx6IZk+YIoBNEzUh1n6KCRocDnra2cmgrjo2TJHqIrT7+c6
   WRhTcQMlHDJlx2JGYCDAE2Mt8KVS/lBBaq2FvJNZBT7uAZgFCKG5FMmfT
   U=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
X-SBRS: 5.1
X-MesageID: 66266747
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:ouqYQq5rWUqr/zfRIxTWpAxRtEfHchMFZxGqfqrLsTDasY5as4F+v
 mcbDGyAO/+IZzb8KNF3Ooq2oBkHupaGztNnSQI5pXgyHi5G8cbLO4+Ufxz6V8+wwmwvb67FA
 +E2MISowBUcFyeEzvuVGuG96yE6j8lkf5KkYAL+EnkZqTRMFWFw0XqPp8Zj2tQy2YPgXlvW0
 T/Pi5a31GGNimYc3l08s8pvmDs31BglkGpF1rCWTakjUG72zxH5PrpGTU2CByKQrr1vNvy7X
 47+IISRpQs1yfuP5uSNyd4XemVSKlLb0JPnZnB+A8BOiTAazsA+PzpS2FPxpi67hh3Q9+2dx
 umhurSiazgtALPll94sdCVjDAU9FPcW3ZP+dC3XXcy7lyUqclPpyvRqSko3IZcZ6qB8BmQmG
 f4wcW5XKErZ3qTvnez9GrIEascLdaEHOKsas2t8zDefDf88X53Sa67L+cVZzHE7gcUm8fP2O
 ZRDN2QwMEuojxtnFHRJErUBpduTpEK8LwN2k1nNj4Yxyj2GpOB2+Oe0a4eEEjCQfu1Kn0Ddv
 nyD82nnDxUyMN2E1SHD4n+qnvXIny7wRMQVDrLQyxJxqATNnCpJUkRQDAbl56ni4qKjZz5BA
 142xS4Qs4E9zn2McuLie16R+i6hkidJDrK8DNYGwA2Kz6PV5SOQCW4FUiNNZbQaiSMmedA5/
 gTXxo20XFSDpJXQECvArenM8VteLABPdQc/iTk4oRzpCjUJiKU6lVrxQ9lqC8ZZZfWlSGirk
 1hmQMXT7oj/bPLnNY3mpTgrYBr2//AlqzLZAC2MBgpJCSsjOOaYi3SAswSz0Bq5BN/xoqO9l
 HYFgdOCy+sFEIuAkieAKM1UQu32vaffaWeM2AQ1d3XEy9hL0yX6FWy3yGsiTHqFz+5eIWO5C
 KMtkV45CGBv0IuCMvYsPtPZ5zUCxqn8D9X1Ps04nfIVCqWdgDSvpXk0DWbJhjiFuBF1zckXZ
 MfKGe7xXC1yIfk2k1KLqxI1jOZDKtYWnjiIG/gWDn2PjNKjWZJiYexcYQXUMbxhsv/sTce82
 483CvZmAi53CYXWSiLW7ZQSPRYNK306DorxsMtZaqiIJQ8OJY3rI6W5LW8JE2C9o5loqw==
IronPort-HdrOrdr: A9a23:jLb0Y61tAvHMk94k43Bm7QqjBRdyeYIsimQD101hICG9Lfb0qy
 n+pp4mPEHP4wr5AEtQ4uxpOMG7MBDhHO1OkPMs1NaZLUPbUQ6TQL2KgrGSpAEIdxeeygcZ79
 YZT0EcMqy8MbEZt7ed3ODQKb9Jr7e6GeKT9J7jJhxWPGNXgtRbnmNE43GgYyhLrWd9ZaYRJd
 653I5qtjCgcXMYYoCQHX8eRdXOoNXNidbPfQMGLwRP0njAsRqYrJrBVzSI1BYXVD1ChZ0493
 LergD/7qK/99mm1x7n0XPJ5Zg+oqqu9jIDPr3MtiEmEESutu+aXvUiZ1REhkFxnAib0idrrD
 ALmWZlAy080QKXQoj/m2qR5+Cp6kdT15al8y7VvZKrm72GeBshF8ROiZhFfhbu41c8tNt81K
 5NwlSCu4BMDRXGqiTxjuK4KC1Chw66p2EvnvUUiGEaWYwCaKVJpYha509NFowcdRiKobzPPd
 Mee/003swmB29yrkqpzFVH0ZipRDA+Dx2GSk8Ntoic1CVXhmlwyw8dyNYElnkN+ZohQ90cjt
 60eZhAhfVLVIsbfKh9DOAOTY++DXHMWwvFNCaXLU78HK8KNnrRo9r84akz5uutZJsUpaFC0K
 jpQRddryo/akjuAcqB0NlC9Q3MWny0WXD3xsRX9/FCy8nBrXrQQFi+oXwV4rudSq8kc7zmst
 6ISeFrP8M=
X-IronPort-AV: E=Sophos;i="5.90,183,1643691600"; 
   d="scan'208";a="66266747"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=KXr+8AObbXwuhCN7JbdJny29FRoX5rlGFe3vF39Pw3P7onHOxKoQwMYsGmgu3fiiHJXrStaPys2WlutTVIuHnzweLNBzZx7NGH8s3KMMxOt+rixsPBo0pG2LgKpN2llYLj5TPCK0PsnC1FA1FU3gXujyHDKISmWnNgC8jIn6KmuF/1JxjYh5IAEdMRBL6cx1M+VvCvbNW9GhwVkfeYM6Vm93HVy5GpfXSiAmeeqH6la9m2+pEcDJuz71t0XuNAHAWIe7c7K20F4Kon++dicoqddtmKmYocP8QO3XHmYc5okF5i3aRq63Ec62yVywTftQ+7ZwfTqsIdrwpZQ/8liY/w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=eETD53zRvhBiQbO7cdvrfJ1RCF19NWp7jHV1MxrkUs0=;
 b=GMoNZSre/z9KoXXuP/iLKlellZr5ibp3V8bIgfdCB/DBOJKeyrbwWYr5P2fcMmwLL218yWtZYv0aqs8m7eM9VClPv8xJYDWOUDitvpSEYqa4XNhEX2jIPmyvVo/k0w6pjjz5VeAfGFcg1vtrdVSHV1sfPwiL2O52eHU9vt73cpT/E8JigDujST8wY1tmou3IHyEo/dS6uImToQuq+lDT7058FKQsDTASmk7HEO0vHsoa7JKv45OOmOIfCGRXo4tLcnYDUWM104n51S+FP9xmbqL4Reo12jcUAD9mlAuJAhNGwJu4I+eolOVP2S861mIl9e40fmji+jcKuCuHKTGKbg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=eETD53zRvhBiQbO7cdvrfJ1RCF19NWp7jHV1MxrkUs0=;
 b=GVnDXomwOCZH3k/5oM/K7ctiHXYRBDX3pKpJ6gnpsLecSN1MLKc/6MM1g9QgCnW9436P7Sj0HpXwC457x3d18JGRtyQkEjnQtRzLbacp2QC+jajX96rS4kImZ2qPx1krezYjNeLJn8cbmSMCxP+62gw6EgMLsJXpgo80fZSZ1zE=
From: Martin Harvey <martin.harvey@citrix.com>
To: Martin Harvey <martin.harvey@citrix.com>,
	"win-pv-devel@lists.xenproject.org" <win-pv-devel@lists.xenproject.org>
Subject: RE: [PATCH] Improve Xenfilt power and default IRP handling.
Thread-Topic: [PATCH] Improve Xenfilt power and default IRP handling.
Thread-Index: AQHYN8VWdc48nzMGh0e89v1joYTp56zATSfg
Date: Tue, 15 Mar 2022 11:18:58 +0000
Message-ID: <MW4PR03MB6347EAF979C17175C7CC4E689A109@MW4PR03MB6347.namprd03.prod.outlook.com>
References: <20220314170141.1565-1-Martin.Harvey@citrix.com>
In-Reply-To: <20220314170141.1565-1-Martin.Harvey@citrix.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 661351f0-9185-445a-09b7-08da067599e7
x-ms-traffictypediagnostic: SJ0PR03MB6630:EE_
x-microsoft-antispam-prvs: <SJ0PR03MB6630E965F0AA375100C965889A109@SJ0PR03MB6630.namprd03.prod.outlook.com>
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: nm7PhiTjQGx/jTLCSfesaw8WjSW1oKevggY0V9QrWlL0b7BuHeG8Ihn7uKhxFa/n7Fuw53PgcV+JNtUHSE20GKl+a2hs2wSoFmEf+Qbh1vJLZJVSRogr8V7cOKc0QsMWGt8DPlEjki8e/RI/DmO42jgQDru3shFklFospzwcDdH6e9AHF/pjb9Qz9vSkEwKw6OzsrXBwxPDHnIb84HADyo0yya7D2Ndd2d6cJC2hni2d80h1qxg9YiWld2o17r8yCXPJz5TD8auJboJYwfAAHTCvUu1ODoehZj5phhL1a3Ykh4iyg0N9WypqkCtJTzyCAay6mVZg+XdiYcS5B2pjtSFRJL9cjWFpbiTscPAW0KFgyiJTkdUkVVFcPrZ3s/Orq+m6PQU52CA5fMHP9KzXN+AZJSXzkwW4UkDhrGELWk70U4QqYdbajCsStPEb+PLosTR7Gu4EYy6BTDayfvvVXkPgpORBQZoB3Gptpg6cQrrUpN7zQr7vu9EKG8UEJbkn7CHyAyGf8yf26s8NUR1pqnP3cwKhWu/QdtxTyE+7A92BtWVH/+xl5OGWBuzZMMuFTISg3kultcxpRTTHMS4SA3cdAFI2gFH3J9ZE8zearJrsHIuhb3mcRfEGzaL4EhYskEGv4fFjCXmG2Ay/F39o794pzr3IzI3JraBL1cEKrkY124JQGGfOqdUYm5fJDb+sW0lP5aKMxxDJH3jX/GO24A==
x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW4PR03MB6347.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(122000001)(508600001)(38100700002)(38070700005)(66476007)(66446008)(71200400001)(66946007)(76116006)(64756008)(110136005)(8676002)(86362001)(7696005)(66556008)(316002)(186003)(26005)(82960400001)(9686003)(83380400001)(6506007)(52536014)(4744005)(5660300002)(8936002)(33656002)(55016003)(44832011)(2906002);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?+9CJ5GJ959sSyqSM+0CEAQdkKro7Rm7I5q0c8u1373TvqoKkFMDiX7+7O5+k?=
 =?us-ascii?Q?+pbHhCR3WDgoudK2xKeZtT1CB4jLYTIzUbozFTDMWq3Qlt60GdHWF6+4dxox?=
 =?us-ascii?Q?DOLrD50GSHe0YQZOpz7Osn+KBBtpGiFPbgzmgic5bKmJes1vVRKI0AhaPdL3?=
 =?us-ascii?Q?8zFg/1P1mPA+BD1WUK4lFJwhBa5dn8ViOjjx246MKzLIXsMJSYXwQ9vP3eQk?=
 =?us-ascii?Q?8hipniOXSabmNSCx5qmo2Z6NnbRv+pMke9Oi+EWjTxnDtpx1WGiiFrKShzzv?=
 =?us-ascii?Q?d2hptXDx7UtMSrPp/qSZDN8X6GqqOCjDPoZF9lmQ1wUlg5x+r7+fwRGtPzoF?=
 =?us-ascii?Q?l19GS88xH6X19s0coiP//wauctxg/3ALzOendzGbT3xPXtFGoobfFY2we4DM?=
 =?us-ascii?Q?Y9J/oEYj+VBcreBhRcsQI8vBb8WeBnkP/8fckMQAj3TBipIIQLCjEf4Z4ykj?=
 =?us-ascii?Q?uL+Q4LVF5l/fMAHM75DT/9jNYhw0IOSb9USQsR91F3TpAeeB2Zc5PbV6b24X?=
 =?us-ascii?Q?SRbZjb4t4W9u1zJG121W8+8Yb6CETpuCae4Hvga6SLTy/mYAibe5zcx1pyQi?=
 =?us-ascii?Q?rolJrWqJ5P7d+QRyJmkxugNU62iHojf+9+QRpqlG70umvSf8y0RR27vc1rti?=
 =?us-ascii?Q?RM2OS4TI9n6kqcVUz1htppqb50VAqp8h0/yyW1f43p5lE2cu+dtiuGlfNrZV?=
 =?us-ascii?Q?hAnhNJ1bq++rg5QcEr8sqjvFL25D0wAteVqjn4YWg8/Bw5aIBTRQHDeDbIE4?=
 =?us-ascii?Q?tk+NqwuuAhW0GgldypO1hQtQaM01V9aEQZx/enmbYiibp6TcUDZW5VCd5mXD?=
 =?us-ascii?Q?HbeyejidNK2FDmKCL1TQgqQg3PhhSZiEwfsolLGkUlJD9Ka06NEm4Ts0pQVy?=
 =?us-ascii?Q?anqQE4rkVZ2x6jfA5DU3D4Hf26ABrOOAoKW06VPCEyIWQ69z+fv/XZnOtIaW?=
 =?us-ascii?Q?Ghc9TjdYNqiu5qnGX3k1Oky/F/N/nsd1ECycSPiZyfSRi2KjlsHeYZJf59uE?=
 =?us-ascii?Q?tW9FzOLo/ekDaZXVKURuB4sNiX3IwOkBN2NL7Wbo2HgaZIbNf7TM/ey3kkTW?=
 =?us-ascii?Q?l729stJptoX1jAptixdFVRkggQrPuCfgkZNblwos6kbCIXZHF/NKT8x+hCy8?=
 =?us-ascii?Q?BrOmlfFRFyQPmyy3IZz8btR4t+lNHXEFP+faui98EcHz0ijIJ8r0/emk07yX?=
 =?us-ascii?Q?33CPJpv6a5KZVoLXkECUOT137nlSoyIzZiomNa7H7GSsUzmh5O4HZdOy3B9k?=
 =?us-ascii?Q?L5e1CGOJvErt5F5QJBUupflaTbnk/+hNoW6Z/ky2veBf95erm+ewFu/zWGLd?=
 =?us-ascii?Q?ANmprA2TCi+j8NQoKjBVQtrfI4syP7nF8nM7eWC6eV9L+HUm3P/FHxpvEVqT?=
 =?us-ascii?Q?xPTpVrrNOSni08SPt8Bsba0oVp9YbYeHY1QtNoExP2Rs4pn74jfnk33p2PLd?=
 =?us-ascii?Q?4GmcoXTL2RUuTzvk3JtRZ/u3FN4l95t1M9DVRnILKh7q9s2O9VubtQ=3D=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MW4PR03MB6347.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 661351f0-9185-445a-09b7-08da067599e7
X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Mar 2022 11:18:58.7779
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: fMNKipXd4qJ5G6EzJLXzkjWTOsKwpgKhdxRCiU0uisFsvoTT7zGqaNWpmesaVFMm9pXS/mR/zxi1SFNijjDyctA6gBAFIF48NPugTXJXFfc=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR03MB6630
X-OriginatorOrg: citrix.com

For context, large patch not submitted on a whim, but fixes occasional inst=
ances of:

DRIVER_POWER_STATE_FAILURE (9f)
A driver has failed to complete a power IRP within a specific time.
Arguments:
Arg1: 0000000000000003, A device object has been blocking an Irp for too lo=
ng a time
Arg2: ffff818002011360, Physical Device Object of the stack
Arg3: fffff8037f46e850, nt!TRIAGE_9F_POWER on Win7 and higher, otherwise th=
e Functional Device Object of the stack
Arg4: ffff81800877d010, The blocked IRP

MH.


From win-pv-devel-bounces@lists.xenproject.org Tue Mar 15 14:01:41 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Tue, 15 Mar 2022 14:01:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.290803.493299 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nU7k4-0007BL-HE; Tue, 15 Mar 2022 14:01:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 290803.493299; Tue, 15 Mar 2022 14:01:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nU7k4-0007BE-EH; Tue, 15 Mar 2022 14:01:40 +0000
Received: by outflank-mailman (input) for mailman id 290803;
 Tue, 15 Mar 2022 14:01:39 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bd58=T2=citrix.com=prvs=066e9a2b4=martin.harvey@srs-se1.protection.inumbo.net>)
 id 1nU7k3-0007B6-9p
 for win-pv-devel@lists.xenproject.org; Tue, 15 Mar 2022 14:01:39 +0000
Received: from esa5.hc3370-68.iphmx.com (esa5.hc3370-68.iphmx.com
 [216.71.155.168]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6bf950b6-a468-11ec-8eba-a37418f5ba1a;
 Tue, 15 Mar 2022 15:01:37 +0100 (CET)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6bf950b6-a468-11ec-8eba-a37418f5ba1a
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1647352897;
  h=from:to:cc:subject:date:message-id:mime-version:
   content-transfer-encoding;
  bh=/friJ6dH8DZn8ViV3B1S7oJzM4wYKf6T3tyKlcxhRDc=;
  b=QAJT9jyMjqdBGkoH6f9LOtQvJsBkOCXIqtfltMkLgC8d92MIDpG+/YR2
   0SpKfXXjIz+pHIt1/c3b4vj6BuUN8A/q47d0v2J3JqFdCcGeWlNW7GYCy
   32gkz6Rz9HO/OEOX4MzXptE8MEF6inEOIX8O3PcAlhq8Iu4PvnV1PdRHB
   s=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
X-SBRS: 5.1
X-MesageID: 65753621
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:pDX4f67Qdkk5eFNuxltKJAxRtF7HchMFZxGqfqrLsTDasY5as4F+v
 mdMD2DVafzcZjegKtAjaoW+9RtU7JfWztVjQAtl+y9nHi5G8cbLO4+Ufxz6V8+wwmwvb67FA
 +E2MISowBUcFyeEzvuVGuG96yE6j8lkf5KkYAL+EnkZqTRMFWFw0XqPp8Zj2tQy2YPgXlvW0
 T/Pi5a31GGNimYc3l08s8pvmDs31BglkGpF1rCWTakjUG72zxH5PrpGTU2CByKQrr1vNvy7X
 47+IISRpQs1yfuP5uSNyd4XemVSKlLb0JPnZnB+A8BOiTAazsA+PzpS2FPxpi67hh3Q9+2dx
 umhurS5ax11A4zGyd87cDxfDCReZ/184e/YdC3XXcy7lyUqclPpyvRqSko3IZcZ6qB8BmQmG
 f4wcW5XKErZ3qTvnez9GrIEascLdaEHOKsas2t8zDefDf88X53Sa67L+cVZzHE7gcUm8fP2O
 ZdGNGExNk2ojxtnYQ0aFbcs2+iS12jlSDtxpGjLhpIu7D2GpOB2+Oe0a4eEEjCQfu1Kn0Ddv
 nyD82nnDxUyMN2E1SHD4n+qnvXIny7wRMQVDrLQyxJxqATNnCpJUkRQDAbl56ni4qKjZz5BA
 0U24HYTjKJjz2qIa97iWj3i8ESd7gFJDrK8DNYGwA2Kz6PV5SOQCW4FUiNNZbQaiSMmedA5/
 gTXxo20XFSDpJXQECvArenM8VteLABPdQc/iTk4oRzpCjUJiKU6lVrxQ9lqC8ZZZfWlSGirk
 1hmQMXT7oj/bPLnNY3mpTgrYBr2//AlqzLZAC2MBgpJCSsjOOaYi3SAswSz0Bq5BN/xoqO9l
 HYFgdOCy+sFEIuAkieAKM1UQu32vaffaWeM2AQ1d3XEy9hL0yX6FWy3yGsiTHqFz+5eIWO5C
 KMtkV45CGBv0IuCMvYsPtPZ5zUCxqn8D9X1Ps04nfIVCqWdgDSvpXk0DWbJhjiFuBF1zckXZ
 MfKGe7xXC1yIfk2k1KLqxI1jOZDKtYWnjiIG/gWDn2PjNKjWZJiYexcYQXUMbxhsv/sTce82
 483CvZmAi53CIXWChQ7O6ZIRbzWBRDX3azLlvE=
IronPort-HdrOrdr: A9a23:199a36uu3WiaPXMctYqehMZG7skDetV00zEX/kB9WHVpmszxra
 6TdZMgpHvJYVcqKRcdcL+7WJVoLUmxyXcX2/h1AV7BZniEhILAFugLhuGO/9SJIVyYygc378
 ZdmsZFZ+EYdWIK7/rH3A==
X-IronPort-AV: E=Sophos;i="5.90,183,1643691600"; 
   d="scan'208";a="65753621"
From: Martin Harvey <Martin.Harvey@citrix.com>
To: <win-pv-devel@lists.xenproject.org>
CC: Martin Harvey <Martin.Harvey@citrix.com>, Martin Harvey
	<martin.harvey@citrix.com>
Subject: [PATCH] Fix div by zero in xenvif if no queues.
Date: Tue, 15 Mar 2022 14:01:00 +0000
Message-ID: <20220315140100.1868-1-Martin.Harvey@citrix.com>
X-Mailer: git-send-email 2.25.0.windows.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

In some cases (live migration), the front/backend can be pending
reconnect at the same point as Tx tries to send packets.
Previously, this caused a div by zero. However, since this is
a transient error, the drivers should ditch the packet,
and carry on.

Signed-off-by: Martin Harvey <martin.harvey@citrix.com>
---
 src/xenvif/frontend.c    | 19 ++++++++++++-------
 src/xenvif/frontend.h    |  5 +++--
 src/xenvif/transmitter.c |  8 +++++++-
 3 files changed, 22 insertions(+), 10 deletions(-)

diff --git a/src/xenvif/frontend.c b/src/xenvif/frontend.c
index e38d2bf..fc4b78c 100644
--- a/src/xenvif/frontend.c
+++ b/src/xenvif/frontend.c
@@ -2174,34 +2174,39 @@ fail1:
     return status;
 }
 
-ULONG
+NTSTATUS
 FrontendGetQueue(
     IN  PXENVIF_FRONTEND                Frontend,
     IN  XENVIF_PACKET_HASH_ALGORITHM    Algorithm,
-    IN  ULONG                           Value
+    IN  ULONG                           Value,
+    OUT PULONG                          QueueIndex
     )
 {
-    ULONG                               Queue;
+    if (QueueIndex == NULL)
+        return STATUS_INVALID_PARAMETER;
 
     switch (Algorithm) {
     case XENVIF_PACKET_HASH_ALGORITHM_NONE:
     case XENVIF_PACKET_HASH_ALGORITHM_UNSPECIFIED:
-        Queue = Value % __FrontendGetNumQueues(Frontend);
+        *QueueIndex = (__FrontendGetNumQueues(Frontend) != 0) ?
+            Value % __FrontendGetNumQueues(Frontend) :
+            0;
         break;
 
     case XENVIF_PACKET_HASH_ALGORITHM_TOEPLITZ:
-        Queue = (Frontend->Hash.Size != 0) ?
+        *QueueIndex = (Frontend->Hash.Size != 0) ?
                 Frontend->Hash.Mapping[Value % Frontend->Hash.Size] :
                 0;
         break;
 
     default:
         ASSERT(FALSE);
-        Queue = 0;
+        *QueueIndex = 0;
         break;
     }
 
-    return Queue;
+    return ((*QueueIndex) < __FrontendGetNumQueues(Frontend)) ?
+        STATUS_SUCCESS : STATUS_INTERNAL_ERROR;
 }
 
 static NTSTATUS
diff --git a/src/xenvif/frontend.h b/src/xenvif/frontend.h
index 8e5552e..5004b0e 100644
--- a/src/xenvif/frontend.h
+++ b/src/xenvif/frontend.h
@@ -237,11 +237,12 @@ FrontendSetHashTypes(
     IN  ULONG               Types
     );
 
-extern ULONG
+extern NTSTATUS
 FrontendGetQueue(
     IN  PXENVIF_FRONTEND                Frontend,
     IN  XENVIF_PACKET_HASH_ALGORITHM    Algorithm,
-    IN  ULONG                           Index
+    IN  ULONG                           Value,
+    OUT PULONG                          QueueIndex
     );
 
 #endif  // _XENVIF_FRONTEND_H
diff --git a/src/xenvif/transmitter.c b/src/xenvif/transmitter.c
index ed89f60..d089301 100644
--- a/src/xenvif/transmitter.c
+++ b/src/xenvif/transmitter.c
@@ -5194,13 +5194,19 @@ TransmitterQueuePacket(
         break;
     }
 
-    Index = FrontendGetQueue(Frontend, Algorithm, Value);
+    status = FrontendGetQueue(Frontend, Algorithm, Value, &Index);
+    if (!NT_SUCCESS(status))
+        goto fail2;
+
     Ring = Transmitter->Ring[Index];
 
     __TransmitterRingQueuePacket(Ring, Packet, More);
 
     return STATUS_SUCCESS;
 
+fail2:
+    Error("fail2\n");
+
 fail1:
     Error("fail1 (%08x)\n", status);
 
-- 
2.25.0.windows.1



From win-pv-devel-bounces@lists.xenproject.org Sat Mar 19 17:37:30 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Sat, 19 Mar 2022 17:37:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.292427.496705 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nVd16-0001eP-6K; Sat, 19 Mar 2022 17:37:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 292427.496705; Sat, 19 Mar 2022 17:37:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nVd16-0001eH-3D; Sat, 19 Mar 2022 17:37:28 +0000
Received: by outflank-mailman (input) for mailman id 292427;
 Sat, 19 Mar 2022 17:37:27 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ky55=T6=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1nVd14-0001e6-VW
 for win-pv-devel@lists.xenproject.org; Sat, 19 Mar 2022 17:37:27 +0000
Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com
 [2a00:1450:4864:20::42d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3c7fae04-a7ab-11ec-8fbc-03012f2f19d4;
 Sat, 19 Mar 2022 18:37:25 +0100 (CET)
Received: by mail-wr1-x42d.google.com with SMTP id u16so14598457wru.4
 for <win-pv-devel@lists.xenproject.org>; Sat, 19 Mar 2022 10:37:24 -0700 (PDT)
Received: from ?IPV6:2a00:23c5:5785:9a01:ad9a:ab78:5748:a7ec?
 ([2a00:23c5:5785:9a01:ad9a:ab78:5748:a7ec])
 by smtp.gmail.com with ESMTPSA id
 v10-20020a056000144a00b00203df06cf9bsm9215636wrx.106.2022.03.19.10.37.22
 for <win-pv-devel@lists.xenproject.org>
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Sat, 19 Mar 2022 10:37:23 -0700 (PDT)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3c7fae04-a7ab-11ec-8fbc-03012f2f19d4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=message-id:date:mime-version:user-agent:reply-to:subject
         :content-language:to:references:from:in-reply-to
         :content-transfer-encoding;
        bh=MXH7C+7gbE5o8yBTlt/FWYk6lxvMcfDBRqPo9E/H3g8=;
        b=D5pCITBmaw93QvG7y78XRuZ1ArK+fEdt75r+DylGanOSz6TXjoLipma4KgtGpZWnol
         Kk9YAaC0Xf8hf8wZnHfHQT6b52VvsoRZhPpp/SgkfHB9ImG8e2e6OlKwulqVYjCJKI3v
         /XM4u5fEnBnRcsXsQqnEM2U+vov0uCbqqsExRTzDRYCJfoeOndQt+jh6Q0yzqwJ/E76y
         xnUgo1GHmnddLSKjFJSa+zp4k9BGKmyoez/Tdzw7qjTJKTlAhfxh9GCtBiOygvWUlyNe
         6vGfk0msw+JJzgbl/voQfi/qOy35qW7ygYzswGf7Z2WxJRfZVBMtgRRp94yDvWoJAsJl
         FL9A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:message-id:date:mime-version:user-agent:reply-to
         :subject:content-language:to:references:from:in-reply-to
         :content-transfer-encoding;
        bh=MXH7C+7gbE5o8yBTlt/FWYk6lxvMcfDBRqPo9E/H3g8=;
        b=IvvJurqWZ8Wc0pWAZRFrnBRUncomnCO1SjQkHpT9cNTEhz1uuoIUih/V2Sfav9M/JZ
         kRAUpTdD/BJncDn2UWaaCDsYIOvp7LH9ecfMQkk8h+rElsLfwEH7WNmj2Dz3rDH1Fg5U
         KCDG7Im+MMu5dDnP7NGLOoK+afL63gBS+Be04/pcWgFIouxzHSZ/BQXKE1MOO4TqjB+n
         F+YF7QhMmIZ+w0WVLSnd/6Ta6P583UGB179wXHYNeqfGno3eCi5X0oSWDL2XzluGbx58
         EzoTLV2qKgOhdBtcMSguDIibEAjQSQLH0TKYjj9fVLQ+yqcq+Dop1yEGQsfvOGbWZiar
         YHQQ==
X-Gm-Message-State: AOAM533AVTdrMQ2VhuiujbzvW69tCV21wH1bAz5IEmDhXQ3U9Bi6hk7a
	2isuQNB7AxEj0iR11l9ANQp6JpAWnQk=
X-Google-Smtp-Source: ABdhPJyaKhGTzbxQhJyvQU9vZUk2SiysX3dh7CjgD/kUZyeyOdXpcCbCapnBrK8IfdHtmhsIbNZwcA==
X-Received: by 2002:adf:90e9:0:b0:204:2ee:7d5 with SMTP id i96-20020adf90e9000000b0020402ee07d5mr2323240wri.536.1647711443489;
        Sat, 19 Mar 2022 10:37:23 -0700 (PDT)
Message-ID: <4dc2d35b-9d73-746e-1ab6-b18d5f75cc66@gmail.com>
Date: Sat, 19 Mar 2022 17:37:22 +0000
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.7.0
Reply-To: paul@xen.org
Subject: Re: [PATCH] Fix div by zero in xenvif if no queues.
Content-Language: en-US
To: win-pv-devel@lists.xenproject.org
References: <20220315140100.1868-1-Martin.Harvey@citrix.com>
From: "Durrant, Paul" <xadimgnik@gmail.com>
In-Reply-To: <20220315140100.1868-1-Martin.Harvey@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 15/03/2022 14:01, Martin Harvey wrote:
> In some cases (live migration), the front/backend can be pending
> reconnect at the same point as Tx tries to send packets.

There is locking that is supposed to stop that from happening. 
VifTransmitterQueuePacketVersion() calls AcquireMrswLockShared() to make 
sure that state is not changing underneath it.

> Previously, this caused a div by zero. However, since this is
> a transient error, the drivers should ditch the packet,
> and carry on.

It really should not happen at all.

   Paul

> 
> Signed-off-by: Martin Harvey <martin.harvey@citrix.com>
> ---
>   src/xenvif/frontend.c    | 19 ++++++++++++-------
>   src/xenvif/frontend.h    |  5 +++--
>   src/xenvif/transmitter.c |  8 +++++++-
>   3 files changed, 22 insertions(+), 10 deletions(-)
> 
> diff --git a/src/xenvif/frontend.c b/src/xenvif/frontend.c
> index e38d2bf..fc4b78c 100644
> --- a/src/xenvif/frontend.c
> +++ b/src/xenvif/frontend.c
> @@ -2174,34 +2174,39 @@ fail1:
>       return status;
>   }
>   
> -ULONG
> +NTSTATUS
>   FrontendGetQueue(
>       IN  PXENVIF_FRONTEND                Frontend,
>       IN  XENVIF_PACKET_HASH_ALGORITHM    Algorithm,
> -    IN  ULONG                           Value
> +    IN  ULONG                           Value,
> +    OUT PULONG                          QueueIndex
>       )
>   {
> -    ULONG                               Queue;
> +    if (QueueIndex == NULL)
> +        return STATUS_INVALID_PARAMETER;
>   
>       switch (Algorithm) {
>       case XENVIF_PACKET_HASH_ALGORITHM_NONE:
>       case XENVIF_PACKET_HASH_ALGORITHM_UNSPECIFIED:
> -        Queue = Value % __FrontendGetNumQueues(Frontend);
> +        *QueueIndex = (__FrontendGetNumQueues(Frontend) != 0) ?
> +            Value % __FrontendGetNumQueues(Frontend) :
> +            0;
>           break;
>   
>       case XENVIF_PACKET_HASH_ALGORITHM_TOEPLITZ:
> -        Queue = (Frontend->Hash.Size != 0) ?
> +        *QueueIndex = (Frontend->Hash.Size != 0) ?
>                   Frontend->Hash.Mapping[Value % Frontend->Hash.Size] :
>                   0;
>           break;
>   
>       default:
>           ASSERT(FALSE);
> -        Queue = 0;
> +        *QueueIndex = 0;
>           break;
>       }
>   
> -    return Queue;
> +    return ((*QueueIndex) < __FrontendGetNumQueues(Frontend)) ?
> +        STATUS_SUCCESS : STATUS_INTERNAL_ERROR;
>   }
>   
>   static NTSTATUS
> diff --git a/src/xenvif/frontend.h b/src/xenvif/frontend.h
> index 8e5552e..5004b0e 100644
> --- a/src/xenvif/frontend.h
> +++ b/src/xenvif/frontend.h
> @@ -237,11 +237,12 @@ FrontendSetHashTypes(
>       IN  ULONG               Types
>       );
>   
> -extern ULONG
> +extern NTSTATUS
>   FrontendGetQueue(
>       IN  PXENVIF_FRONTEND                Frontend,
>       IN  XENVIF_PACKET_HASH_ALGORITHM    Algorithm,
> -    IN  ULONG                           Index
> +    IN  ULONG                           Value,
> +    OUT PULONG                          QueueIndex
>       );
>   
>   #endif  // _XENVIF_FRONTEND_H
> diff --git a/src/xenvif/transmitter.c b/src/xenvif/transmitter.c
> index ed89f60..d089301 100644
> --- a/src/xenvif/transmitter.c
> +++ b/src/xenvif/transmitter.c
> @@ -5194,13 +5194,19 @@ TransmitterQueuePacket(
>           break;
>       }
>   
> -    Index = FrontendGetQueue(Frontend, Algorithm, Value);
> +    status = FrontendGetQueue(Frontend, Algorithm, Value, &Index);
> +    if (!NT_SUCCESS(status))
> +        goto fail2;
> +
>       Ring = Transmitter->Ring[Index];
>   
>       __TransmitterRingQueuePacket(Ring, Packet, More);
>   
>       return STATUS_SUCCESS;
>   
> +fail2:
> +    Error("fail2\n");
> +
>   fail1:
>       Error("fail1 (%08x)\n", status);
>   



From win-pv-devel-bounces@lists.xenproject.org Sat Mar 19 17:39:27 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Sat, 19 Mar 2022 17:39:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.292434.496708 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nVd31-0001lN-C2; Sat, 19 Mar 2022 17:39:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 292434.496708; Sat, 19 Mar 2022 17:39:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nVd31-0001lG-9G; Sat, 19 Mar 2022 17:39:27 +0000
Received: by outflank-mailman (input) for mailman id 292434;
 Sat, 19 Mar 2022 17:39:26 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ky55=T6=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1nVd2z-0001lA-RQ
 for win-pv-devel@lists.xenproject.org; Sat, 19 Mar 2022 17:39:25 +0000
Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com
 [2a00:1450:4864:20::32b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 83575477-a7ab-11ec-a405-831a346695d4;
 Sat, 19 Mar 2022 18:39:24 +0100 (CET)
Received: by mail-wm1-x32b.google.com with SMTP id
 bi13-20020a05600c3d8d00b0038c2c33d8f3so5593692wmb.4
 for <win-pv-devel@lists.xenproject.org>; Sat, 19 Mar 2022 10:39:22 -0700 (PDT)
Received: from ?IPV6:2a00:23c5:5785:9a01:ad9a:ab78:5748:a7ec?
 ([2a00:23c5:5785:9a01:ad9a:ab78:5748:a7ec])
 by smtp.gmail.com with ESMTPSA id
 i35-20020adf90a6000000b00203e767a1d2sm7192153wri.103.2022.03.19.10.39.21
 for <win-pv-devel@lists.xenproject.org>
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Sat, 19 Mar 2022 10:39:21 -0700 (PDT)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 83575477-a7ab-11ec-a405-831a346695d4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=message-id:date:mime-version:user-agent:reply-to:subject
         :content-language:to:references:from:in-reply-to
         :content-transfer-encoding;
        bh=ZVN/MBRih7v28fpb+4I+m+719FIvoF+tPacwSK/XuV8=;
        b=knnnMGNsX4xlV5LZ8LYdv2xpcFCjE8pgT1Pn7RTjlp2OWk9eYE+Q/ord0SB7UDfdQT
         FM7gk8nOTo8cM3Q9Xnu6rPdl9i6c3sD/1z//l0nP5PZ/n/gUxZSJCPvtXrI/5EALqhGv
         Zo3PbPk3Gy4G3NmxCQUEi/izPYXtH+R4sECaE3nfbnw+MSMtTLUbzzhMXpOy8BbKgsdH
         MydKuW+K6wHMXrIfo1lV108iffuFW4x3U7r4gaQU59OyQsuPGpczrqAuTk6IHzm2qD10
         jCl5flFLNvjrwpOog4hEv3+rqVpOtUFUZWMH2OtFNuYJqWP21b7hFatcEssEoYC1YLtg
         2faA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:message-id:date:mime-version:user-agent:reply-to
         :subject:content-language:to:references:from:in-reply-to
         :content-transfer-encoding;
        bh=ZVN/MBRih7v28fpb+4I+m+719FIvoF+tPacwSK/XuV8=;
        b=OBSkxlgpwJefuuUxkhpri+/LUnrtUAJ9Gb1rekbXnpT8yNtN8uHkTpPLJfiUkhj26f
         weQLsg3Pyo87b5XTIP4ELJuUc1FwCevBr00svuKBU8TZ+eHRatZ1NoTBf7BRSmVHcgoQ
         s9tExJtIl0pW18vt2Q0KtxKfHpNq63XTpZtyoGCy1gpgDOLhKsGQ93/RukZQ3HnJQo6I
         lqehBR1VM/Rk6muyaPy43X1Cgf16gebCVOhPAHyhV9qVyKj+hqk9REbRBz272SwtuYdE
         9LQ8P9h9PvwA69j1gK999lgMYj8wv9SJqcW2Zy8o3gmAt0MNOJagLF9FsOc8Djc5ZT/4
         3JNA==
X-Gm-Message-State: AOAM5325asfmzyc5yS3DyZTKSW/mUvE4hL8WuVfndwkxyVTBHyufLvcl
	9YtxDvRALwOAqNKjpKqE8lFn5Y0XIcg=
X-Google-Smtp-Source: ABdhPJz76mhXmDVMliyEtDN4hBQnmv82be649PeNhEPFhduE7X1etVZeJ3+GAs4MAkj56l4b4sOhDQ==
X-Received: by 2002:a05:600c:190c:b0:389:bf36:ce3c with SMTP id j12-20020a05600c190c00b00389bf36ce3cmr20344508wmq.155.1647711562213;
        Sat, 19 Mar 2022 10:39:22 -0700 (PDT)
Message-ID: <e81b19af-067b-1a98-7952-f06c101a2418@gmail.com>
Date: Sat, 19 Mar 2022 17:39:21 +0000
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.7.0
Reply-To: paul@xen.org
Subject: Re: [PATCH] Windows PV drivers fail to set up RSS when vCPUs > 8
Content-Language: en-US
To: win-pv-devel@lists.xenproject.org
References: <20220314152746.1712-1-Martin.Harvey@citrix.com>
From: "Durrant, Paul" <xadimgnik@gmail.com>
In-Reply-To: <20220314152746.1712-1-Martin.Harvey@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 14/03/2022 15:27, Martin Harvey wrote:

I think we need a bit more in the commit comment. What is the nature of 
the failure... and does XENNET advertise more than 8 queues, so will the 
situation ever arise? Linux certainly tops out at 8 queues.

   Paul

> Signed-off-by: Martin Harvey <Martin.Harvey@citrix.com>
> ---
>   src/xenvif/receiver.c | 168 +++++++++++++++++++++++++++++++++++++-----
>   1 file changed, 148 insertions(+), 20 deletions(-)
> 
> diff --git a/src/xenvif/receiver.c b/src/xenvif/receiver.c
> index 10ac6f5..0c7b32a 100644
> --- a/src/xenvif/receiver.c
> +++ b/src/xenvif/receiver.c
> @@ -106,6 +106,7 @@ typedef struct _XENVIF_RECEIVER_RING {
>       PLIST_ENTRY                 PacketQueue;
>       KDPC                        QueueDpc;
>       ULONG                       QueueDpcs;
> +    PROCESSOR_NUMBER            TargetProcessor;
>       LIST_ENTRY                  PacketComplete;
>       XENVIF_RECEIVER_HASH        Hash;
>   } XENVIF_RECEIVER_RING, *PXENVIF_RECEIVER_RING;
> @@ -2498,6 +2499,9 @@ __ReceiverRingInitialize(
>   
>       KeInitializeThreadedDpc(&(*Ring)->QueueDpc, ReceiverRingQueueDpc, *Ring);
>   
> +    status = KeGetProcessorNumberFromIndex((*Ring)->Index, &(*Ring)->TargetProcessor);
> +    ASSERT(NT_SUCCESS(status));
> +
>       return STATUS_SUCCESS;
>   
>   fail7:
> @@ -2550,6 +2554,45 @@ fail1:
>       return status;
>   }
>   
> +static NTSTATUS
> +__ReceiverRingSetAffinity(
> +    IN  PXENVIF_RECEIVER_RING   Ring,
> +    IN  PPROCESSOR_NUMBER       Processor
> +    )
> +{
> +    PXENVIF_RECEIVER            Receiver;
> +    PXENVIF_FRONTEND            Frontend;
> +    NTSTATUS status;
> +
> +    status = STATUS_INVALID_PARAMETER;
> +    if ((Ring == NULL) || (Processor == NULL))
> +        goto fail1;
> +
> +    Receiver = Ring->Receiver;
> +    Frontend = Receiver->Frontend;
> +
> +    /* Always update ring target processor
> +       Actually set affinities if frontend override not present.
> +       Re-bind event-channel if already connected */
> +
> +    __ReceiverRingAcquireLock(Ring);
> +
> +    Ring->TargetProcessor = *Processor;
> +
> +    /* Don't rebind event channel at this point. */
> +    KeSetTargetProcessorDpcEx(&Ring->PollDpc, &Ring->TargetProcessor);
> +    KeSetTargetProcessorDpcEx(&Ring->QueueDpc, &Ring->TargetProcessor);
> +
> +    __ReceiverRingReleaseLock(Ring);
> +
> +    return STATUS_SUCCESS;
> +
> +fail1:
> +    Error("fail1 (%08x)\n", status);
> +
> +    return status;
> +}
> +
>   static FORCEINLINE NTSTATUS
>   __ReceiverRingConnect(
>       IN  PXENVIF_RECEIVER_RING   Ring
> @@ -2560,7 +2603,6 @@ __ReceiverRingConnect(
>       PFN_NUMBER                  Pfn;
>       CHAR                        Name[MAXNAMELEN];
>       ULONG                       Index;
> -    PROCESSOR_NUMBER            ProcNumber;
>       NTSTATUS                    status;
>   
>       Receiver = Ring->Receiver;
> @@ -2637,16 +2679,17 @@ __ReceiverRingConnect(
>       if (Ring->Channel == NULL)
>           goto fail6;
>   
> -    status = KeGetProcessorNumberFromIndex(Ring->Index, &ProcNumber);
> -    ASSERT(NT_SUCCESS(status));
> +    status = XENBUS_EVTCHN(Bind,
> +                            &Receiver->EvtchnInterface,
> +                            Ring->Channel,
> +                            Ring->TargetProcessor.Group,
> +                            Ring->TargetProcessor.Number);
> +    if (!NT_SUCCESS(status))
> +        Warning("Cound not set initial receiver ring affinity: 0x%x\n", status);
> +    /* You haven't specifically asked for an affinity yet, so just warn. */
>   
> -    KeSetTargetProcessorDpcEx(&Ring->PollDpc, &ProcNumber);
> -
> -    (VOID) XENBUS_EVTCHN(Bind,
> -                         &Receiver->EvtchnInterface,
> -                         Ring->Channel,
> -                         ProcNumber.Group,
> -                         ProcNumber.Number);
> +    KeSetTargetProcessorDpcEx(&Ring->PollDpc, &Ring->TargetProcessor);
> +    KeSetTargetProcessorDpcEx(&Ring->QueueDpc, &Ring->TargetProcessor);
>   
>       (VOID) XENBUS_EVTCHN(Unmask,
>                            &Receiver->EvtchnInterface,
> @@ -2665,11 +2708,6 @@ __ReceiverRingConnect(
>       if (!NT_SUCCESS(status))
>           goto fail7;
>   
> -    status = KeGetProcessorNumberFromIndex(Ring->Index, &ProcNumber);
> -    ASSERT(NT_SUCCESS(status));
> -
> -    KeSetTargetProcessorDpcEx(&Ring->QueueDpc, &ProcNumber);
> -
>       return STATUS_SUCCESS;
>   
>   fail7:
> @@ -3917,6 +3955,56 @@ fail1:
>       return status;
>   }
>   
> +static NTSTATUS
> +__ReceiverSetQueueAffinities(
> +    IN  PXENVIF_RECEIVER        Receiver,
> +    IN  PPROCESSOR_NUMBER       QueueAffinities,
> +    IN  ULONG                   Count
> +    )
> +{
> +    PXENVIF_FRONTEND        Frontend;
> +    ULONG                   Index;
> +    NTSTATUS                status;
> +    KIRQL                   Irql;
> +
> +    Frontend = Receiver->Frontend;
> +
> +    status = STATUS_INVALID_PARAMETER;
> +
> +    if (QueueAffinities == NULL)
> +        goto fail1;
> +
> +    if (Count > FrontendGetNumQueues(Frontend))
> +        goto fail2;
> +
> +    KeRaiseIrql(DISPATCH_LEVEL, &Irql);
> +
> +    for (Index = 0; Index < Count; Index++) {
> +        PXENVIF_RECEIVER_RING   Ring = Receiver->Ring[Index];
> +
> +        status = __ReceiverRingSetAffinity(Ring, &QueueAffinities[Index]);
> +        if (!NT_SUCCESS(status))
> +            goto fail3;
> +    }
> +
> +    KeLowerIrql(Irql);
> +
> +    return STATUS_SUCCESS;
> +
> +fail3:
> +    KeLowerIrql(Irql);
> +
> +    Error("fail3\n");
> +
> +fail2:
> +    Error("fail2\n");
> +
> +fail1:
> +    Error("fail1 (%08x)\n", status);
> +
> +    return status;
> +}
> +
>   NTSTATUS
>   ReceiverUpdateHashMapping(
>       IN  PXENVIF_RECEIVER    Receiver,
> @@ -3926,10 +4014,15 @@ ReceiverUpdateHashMapping(
>   {
>       PXENVIF_FRONTEND        Frontend;
>       PULONG                  QueueMapping;
> +    PPROCESSOR_NUMBER       QueueAffinities;
>       ULONG                   NumQueues;
> +    ULONG                   QueuesDetermined;
> +    ULONG                   QIndex;
>       ULONG                   Index;
> +    BOOLEAN                 MapEntryDone;
>       NTSTATUS                status;
>   
> +
>       Frontend = Receiver->Frontend;
>   
>       QueueMapping = __ReceiverAllocate(sizeof (ULONG) * Size);
> @@ -3939,26 +4032,61 @@ ReceiverUpdateHashMapping(
>           goto fail1;
>   
>       NumQueues = FrontendGetNumQueues(Frontend);
> +    QueuesDetermined = 0;
> +
> +    QueueAffinities = __ReceiverAllocate(sizeof(PROCESSOR_NUMBER) * NumQueues);
> +    if (QueueAffinities == NULL)
> +        goto fail2;
>   
>       status = STATUS_INVALID_PARAMETER;
> +    /* N^Squared-ish, but performed infrequently */
>       for (Index = 0; Index < Size; Index++) {
> -        QueueMapping[Index] = KeGetProcessorIndexFromNumber(&ProcessorMapping[Index]);
> -
> -        if (QueueMapping[Index] >= NumQueues)
> -            goto fail2;
> +        MapEntryDone = FALSE;
> +        /* Existing queue meets affinity requirement for the mapping at this index? */
> +        for (QIndex = 0; QIndex < QueuesDetermined; QIndex++) {
> +            if ((QueueAffinities[QIndex].Group == ProcessorMapping[Index].Group) &&
> +                (QueueAffinities[QIndex].Number == ProcessorMapping[Index].Number)) {
> +                QueueMapping[Index] = QIndex;
> +                MapEntryDone = TRUE;
> +            }
> +        }
> +        if (!MapEntryDone) {
> +            /* New queue "allocation", with new affinity, if possible */
> +            if (QueuesDetermined < NumQueues) {
> +                QIndex = QueuesDetermined;
> +                QueueAffinities[QIndex] = ProcessorMapping[Index];
> +                QueueMapping[Index] = QIndex;
> +                QueuesDetermined ++;
> +            } else {
> +                goto fail3;
> +            }
> +        }
>       }
>   
>       status = FrontendSetHashMapping(Frontend, QueueMapping, Size);
>       if (!NT_SUCCESS(status))
> -        goto fail3;
> +        goto fail4;
> +
> +    status = __ReceiverSetQueueAffinities(Receiver, QueueAffinities, QueuesDetermined);
> +    if (!NT_SUCCESS(status))
> +        goto fail5;
>   
>       __ReceiverFree(QueueMapping);
> +    __ReceiverFree(QueueAffinities);
>   
>       return STATUS_SUCCESS;
>   
> +fail5:
> +    Error("fail5\n");
> +
> +fail4:
> +    Error("fail4\n");
> +
>   fail3:
>       Error("fail3\n");
>   
> +    __ReceiverFree(QueueAffinities);
> +
>   fail2:
>       Error("fail2\n");
>   



From win-pv-devel-bounces@lists.xenproject.org Mon Mar 21 14:23:21 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Mon, 21 Mar 2022 14:23:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.293020.497695 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nWIwJ-0003aS-Tr; Mon, 21 Mar 2022 14:23:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 293020.497695; Mon, 21 Mar 2022 14:23:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nWIwJ-0003aK-Qq; Mon, 21 Mar 2022 14:23:19 +0000
Received: by outflank-mailman (input) for mailman id 293020;
 Mon, 21 Mar 2022 14:23:18 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nBRh=UA=citrix.com=prvs=07264f618=owen.smith@srs-se1.protection.inumbo.net>)
 id 1nWIwI-0003aE-SA
 for win-pv-devel@lists.xenproject.org; Mon, 21 Mar 2022 14:23:18 +0000
Received: from esa6.hc3370-68.iphmx.com (esa6.hc3370-68.iphmx.com
 [216.71.155.175]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 719640e6-a922-11ec-a405-831a346695d4;
 Mon, 21 Mar 2022 15:23:16 +0100 (CET)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 719640e6-a922-11ec-a405-831a346695d4
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1647872596;
  h=from:to:cc:subject:date:message-id:mime-version:
   content-transfer-encoding;
  bh=gHyr/xGJdptY8JafFiU9ULEbUqktszbtM85sWaJBx88=;
  b=Y57kn3k3WE+3p7FUjvwTzotpkstYj1UhxC0EGFILo9+uxpc+DgLFrcV6
   6TOwmyvgO51ZBlKtipXwD90a+ynmma2BFrMDeRYGeJCvrIjlrGOYpYhcD
   kZtTJ8mo5O5TibKP8zGM3zS0RCFFfZ0WU5d+yg5RT+ElG2YnvRA01U2b8
   g=;
Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
X-SBRS: 5.1
X-MesageID: 66688104
X-Ironport-Server: esa6.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:yh7m46MvltG1ZKfvrR30l8FynXyQoLVcMsEvi/4bfWQNrUor1jUGn
 2NOX2iPOfaMa2T8eN9/b4qy9BtUsZDcn4AwQQto+SlhQUwRpJueD7x1DKtR0wB+jCHnZBg6h
 ynLQoCYdKjYdleF+lH1dOKJQUBUjclkfJKlYAL/En03FFcMpBsJ00o5wbZi2Ncw2LBVPivW0
 T/Mi5yHULOa82Yc3lI8s8pvfzs24ZweEBtB1rAPTagjUG32zhH5P7pGTU2FFFPqQ5E8IwKPb
 72rIIdVXI/u10xF5tuNyt4Xe6CRK1LYFVDmZnF+A8BOjvXez8CbP2lS2Pc0MC9qZzu1c99Z8
 MV9pbC+c1wSMJLlntw0a0dINSdGMvgTkFPHCSDXXc27ykTHdz3nwul0DVFwNoodkgp1KTgQr
 7pCcmlLN03dwbLtqF64YrAEasALK9bmOsUEv3B8zRnSDOo8QICFSKLPjTNd9Glu2JAeRK6CD
 yYfQRBxdR7ESEx9A1xNWaNgxsKQwULhQiIN/Tp5ooJoujOOnWSdyoPFL9vTP8SUAMlYgEucj
 mbH5HjiRAEXMsSFzjiI+W7qgfXA9R4XQ6pLSuf+rKQzxgTOmCpDU3X6SGdXv9GBmn6ud9xlc
 HcaxQMolYEKqlyNSOHyCkjQTGG/gjYQXN9ZEusf4Q6Ly7bJ7wvxOlXoXgKte/R96pZoGGVCO
 kuh2oqwWGcx6OH9pWe1rO/8kN+kBcQCwYbujwcgRBBN3dTsqZpbYvnnHoc6S/7dYjEY9FjNL
 9G2QMoW2u57YS0jjfzTEbX7b9WE/MShc+LNzl+LNl9JFysgDGJfW6Sm6ELA8dFLJ5uDQ1+Ks
 RAswpbCs7BXVcjUxHXWEY3h+Y1FAN7fbVXhbaNHRcF9p1xBBVb9FWyv3N2ODBgwaZtVEdMYS
 ETSpRlQ9Pdu0IiCNsdKj3aKI51yl8DITI29PtiNN4YmSsUhJWevoXA1DWbNjj+FraTZufxmU
 XttWZ33Vihy5GUO5GfeetrxJpdwn3FgnzyPHc6np/lluJLHDEOopX4+GAPmRogEAGms+W05L
 /432xO29ihi
IronPort-HdrOrdr: A9a23:HSWt/qkrf/KAL7d+H7C3KDYV+rfpDfIm3DAbv31ZSRFFG/Fxl6
 iV88jzsiWE7Qr5OUtQ/uxoV5PgfZqxz/NICOoqTNWftWvd2FdARbsKheCJ/9SJIVybygc378
 ldmsZFZOEYdWIK7vrH3A==
X-IronPort-AV: E=Sophos;i="5.90,198,1643691600"; 
   d="scan'208";a="66688104"
From: Owen Smith <owen.smith@citrix.com>
To: <win-pv-devel@lists.xenproject.org>
CC: Owen Smith <owen.smith@citrix.com>
Subject: [PATCH] Use STATUS_PNP_REBOOT_REQUIRED in IRP_MN_START_DEVICE when a reboot is required
Date: Mon, 21 Mar 2022 14:23:07 +0000
Message-ID: <20220321142307.114-1-owen.smith@citrix.com>
X-Mailer: git-send-email 2.33.0.windows.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

Failing IRP_MN_START_DEVICE can lead to WHQL distribution of drivers to be
rejected by MS, as too many VMs in the gradual rollout phase will report this
error on update. Using STATUS_PNP_REBOOT_REQUIRED will indicate that the VM
requires a reboot, and should not be treated as a gating factor (it should also
trigger the Windows Update process to report the reboot to users and/or trigger
an automatic reboot)

Signed-off-by: Owen Smith <owen.smith@citrix.com>
---
 src/xenvif/pdo.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/xenvif/pdo.c b/src/xenvif/pdo.c
index bc9b9ed..b3a4aae 100644
--- a/src/xenvif/pdo.c
+++ b/src/xenvif/pdo.c
@@ -1303,7 +1303,7 @@ PdoStartDevice(
     if (Pdo->HasAlias) {
         PdoUnplugRequest(Pdo, TRUE);
 
-        status = STATUS_UNSUCCESSFUL;
+        status = STATUS_PNP_REBOOT_REQUIRED;
         goto fail9;
     }
 
-- 
2.33.0.windows.2



From win-pv-devel-bounces@lists.xenproject.org Mon Mar 21 16:07:06 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Mon, 21 Mar 2022 16:07:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.293072.497809 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nWKYj-0002RN-3x; Mon, 21 Mar 2022 16:07:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 293072.497809; Mon, 21 Mar 2022 16:07:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nWKYj-0002RG-0w; Mon, 21 Mar 2022 16:07:05 +0000
Received: by outflank-mailman (input) for mailman id 293072;
 Mon, 21 Mar 2022 16:07:04 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=pDjm=UA=citrix.com=prvs=0726b29d2=martin.harvey@srs-se1.protection.inumbo.net>)
 id 1nWKYh-0002RA-Mm
 for win-pv-devel@lists.xenproject.org; Mon, 21 Mar 2022 16:07:03 +0000
Received: from esa2.hc3370-68.iphmx.com (esa2.hc3370-68.iphmx.com
 [216.71.145.153]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id eec08180-a930-11ec-8fbc-03012f2f19d4;
 Mon, 21 Mar 2022 17:07:00 +0100 (CET)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eec08180-a930-11ec-8fbc-03012f2f19d4
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1647878820;
  h=from:to:cc:subject:date:message-id:references:
   in-reply-to:content-transfer-encoding:mime-version;
  bh=9jVF6ZxvD3ef0GPYq2jYETcJmU0msUnALv1Xy2TqPbc=;
  b=I62BCGsmx7myW5J2s6S4/+fsVnltADNrDvzkhlJfRXPOwBiZcU8sdcnJ
   T39vFBQknasmki4rasaR6RoY8rCU3SYL7sMdqHpaAH2lFwa80DHTxKU5l
   OYRUsFrfqu2eb786xyRRx/+YCQtrseB48oB0Vc5whEJ+2QL0exo7lOo19
   4=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
X-SBRS: 5.1
X-MesageID: 66786094
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:7uP/mq2xNrlZ5gfDb/bD5dNxkn2cJEfYwER7XKvMYLTBsI5bpzcOz
 WROC27QP/eOMWWmet13bYzi80wHupPQzIVjSFM9pC1hF35El5HIVI+TRqvS04J+DSFhoGZPt
 Zh2hgzodZhsJpPkjk7xdOCn9xGQ7InQLlbGILes1htZGEk1EE/NtTo5w7Rj2tUy2IDja++wk
 YiaT/P3aQfNNwFcagr424rbwP+4lK2v0N+wlgVWicFj5DcypVFMZH4sDfjZw0/DaptVBoaHq
 9Prl9lVyI97EyAFUbtJmp6jGqEDryW70QKm0hK6UID66vROS7BbPg/W+5PwZG8O4whlkeydx
 /1K5JvoQFcXb5HcleA+elpgTzF7PKxJreqvzXiX6aR/zmXDenrohf5vEFs3LcsT/eMf7WNmr
 KJCbmpXN1ba2rzwkOnTpupE36zPKOHuNZkDu3cmzTjDE/s3aZvCX7/L9ZlT2zJYasVmQ6iBN
 5RBOGYHgBLoZgRwOQpQNtEEwe6nnUbTfxB+r3a6jP9ii4TU5FMoi+W8WDbPQfSVRMMQhljdq
 m/Y8mDRBhABKMfZ2TeD6mirhOLEgWX8Qo16PKWx6/pCkFCVgGsJB3UruUCT+KfjzBTkApQGd
 hJSqnFGQbUOGFKDSYj6BxToni+4rhMYePMTK7Eq7QCMxf+Bi+qGPVQsQjlEYd0gkcY5Qz02y
 1OE9+/U6SxTXK69EizEqOrNxd+mEW1MdDJZO3dYJecQy4O7yLzfmC4jWTqK/ESdqtTuUQ/9z
 DmRxMTVr+VC1JVbv0lXEL2uvt5NmnQrZlNujuk0djj8hu+cWGJCT9b0gWU3Fd4acO6koqO9l
 HYFgdOCy+sFEIuAkieAKM1UQu32t6/daWSM2AU+d3XEy9hK0yT+Fb28HRkkfBs5Wir6UWGBj
 LDvVfN5u8YIYSrCgV5faIOtEcU6pZUM5vy+Ps04muFmO8ArHCfepXkGTRfJgwjFzRh9+Ylia
 MzzWZv9Uh4n5VFPkWPeb/0DyoUi2i1W7TqVHfgXOTz8iuHADJNUIJ9YWGazghcRt/rV/l+Kr
 44Ba6NnCXx3CYXDX8UeyqZKRXgiJnknH5Hm7ctRc++IOA19H289TfTWxNscl0ZNxMy5Ss+gE
 qmBZ3Jl
IronPort-HdrOrdr: A9a23:T9RSy6sXBY2wHS9+/h+IrQyF7skCxIMji2hC6mlwRA09TyXGra
 6TdaUguiMc1gx8ZJh5o6H9BEGBKUmskaKdkrNhQotKOzOW91dATbsSobcKpgeAJ8SQzJ8k6U
 4NSdkdNDS0NykGsS+Y2nj3Lz9D+qj9zEnAv463pB0BLXAIV0gj1XYCNu/yKDwveOAsP+tfKH
 Po3Ls/m9PWQwVwUi3UPAhhY8Hz4/nw0L72ax8PABAqrCOUiymz1bL8Gx+Emj8DTjJm294ZgC
 n4uj28wp/mn+Cwyxfa2WOWxY9RgsHdxtxKA9HJotQJKw/rlh2jaO1aKv2/VXEO0aKSAWQR4Z
 zxSiQbToBOArTqDyaISC7WqkvdOfAVmjnfIBGj8CLeSIfCNUwH4oJ69PNkm13imhEdVZhHod
 J29nPcuJxNARzamiPho9DOShFxj0Kx5WEviOgJkhVkIMAjgZJq3MUiFXluYd899ePBmfQaOf
 grCNuZ6OddcFucYXyctm5zwMa0VnB2GhudWEANtsGczjATxRlCvgYl7d1amm1F+IM2SpFC6e
 iBOqN0lKtWRstTaa5mHu8OTca+F2SISxPRN2CZJ0jhCcg8Sjjwgo+y5K9w6PCheZQOwpd3kJ
 PdUElAvWp3YE7qAd3m5uw8zvkMehTLYd3A8LAt23EigMyOeFPCC1z3dGwT
X-IronPort-AV: E=Sophos;i="5.90,199,1643691600"; 
   d="scan'208";a="66786094"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=d9GLdOfDe/mckvdg0z2vqmCDHUevY3MNi8Ya9ZnCTVlNV/6h06h8IL27aM1AMrwkn//3yiY3Pl8wzo5pb5EIUOpTsjndJYI/VJoEnynil7jhWnL0RnRSW1UrNB39PvCmKAck4C2SIpzsPc4CSN4G5mXQyKw+lgcN+tx/72HPK/P9wjenMwo8ur2oa4Kyo7VAJpLYzMavt6pywoUEWYQ8AWHLZ+D1YPtfTlsmoScWPC6J8E5MzOOhe7UqXjhDQEx9gOmtL/fEKsavcFUn/j5Ky9dkcoP8plFn4g1t5eDaFelzbkChikc6YcHhGRD0NLQO8khttHURWsNSf+sLISAdNA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=9jVF6ZxvD3ef0GPYq2jYETcJmU0msUnALv1Xy2TqPbc=;
 b=U9//e/Btls+XI964sAmG/EbI4RUWriNxHnie3DvYCK6AjevNYS2zGcK9QWoMBladvjpje4UZRfQIckg51ahWFnKBcFDt0YRTAy9+aQO3HD5lq3pt7czgEf6KNYNp/gRc7EFZuu6QvXkPU/cJ3XvPcpJeAfRp7UQX23cOYIuRusD78xkciC/zKANCUlknH4GL9cfm8IXZLoNIltF35zvh+7PWPklaQkCO9xQJ2GbR6oacMUu96piwSJqe0uRR7u+uJ/zSoiktjyGdZz/Rj3Q7kUDe7RTRyIck90rqa8DMvHdiCWKIkZPH7kffNE6iLyE5LirpjMIyYznkvkkHcz2Ikg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=9jVF6ZxvD3ef0GPYq2jYETcJmU0msUnALv1Xy2TqPbc=;
 b=c1iNCE4d5I4Mk3FR5JFEDJbk9UNut+QDCVizerEfZzDnFtPFdMROQ8xuJLT5DsmggIbyMsnA9LzuVWeMWbOejgY/SH00zFVOsTGjiwFrn+GVqonZ+7EtaC7BeTv7XGJa1KEU80hEfwuxuWIW+RtA9loHN5xMvSxi295074jXqXc=
From: Martin Harvey <martin.harvey@citrix.com>
To: "paul@xen.org" <paul@xen.org>, "win-pv-devel@lists.xenproject.org"
	<win-pv-devel@lists.xenproject.org>
CC: Edwin Torok <edvin.torok@citrix.com>
Subject: RE: [PATCH] Windows PV drivers fail to set up RSS when vCPUs > 8
Thread-Topic: [PATCH] Windows PV drivers fail to set up RSS when vCPUs > 8
Thread-Index: AQHYN7gtXmzUE2l3x0isfDYfnRWjD6zHAReAgAMJTjA=
Date: Mon, 21 Mar 2022 16:06:52 +0000
Message-ID: <MW4PR03MB63476907571ADA5598BE441E9A169@MW4PR03MB6347.namprd03.prod.outlook.com>
References: <20220314152746.1712-1-Martin.Harvey@citrix.com>
 <e81b19af-067b-1a98-7952-f06c101a2418@gmail.com>
In-Reply-To: <e81b19af-067b-1a98-7952-f06c101a2418@gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 50c8a9eb-d963-4a62-7500-08da0b54d04d
x-ms-traffictypediagnostic: BN7PR03MB3620:EE_
x-microsoft-antispam-prvs: <BN7PR03MB3620D3EC8A232D112CE782BA9A169@BN7PR03MB3620.namprd03.prod.outlook.com>
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 4xRwOSb1UqjJe90YHoJpd5WOt4LGt+KdVZG5dqo0KiTkMKkHnkQeovAvyGCcm8p7FGlFdvjnAkiFSQqoU4jC5IHow88H97YaRal0OeEjYLLNq03TYLjUPaDYgDc0jV1iwzgduVkV1zA+SrswRqShVO+jfRNK7LzrWd3DFwFiQjzZJ+KOgGyrLeWFwV40h0xW2QJgoeA+2K7RvRfPm7TCM8nBWQuKFvuUyldLF9kE5nWR3zm7W482d2I4Nyjth58MRTynE+KsuB0hBn/xV0uGdDd6dqQDZTjdcDKEhyvBtiVIFNkoToaf185TLANB66YqyKkeVE/pOh75TImCpyP3kO+b12MKpDTUV7k1wIy+7sCiKDtMf3Z0rrKRmNz7Ri7JSYVybewftPxfQtEWuYxCE54dOIuP1m7h9Cxcbn6Yu+gHv8l8Bxwkwy9TqAAuzOYcS8ogE/He/mbNgW48EcrWIKRu3/2k7nihq5JwfbwUNWQ5m+KQnYayLbT9rRHlV5F6VjXOEKiS0sRa0elsSG0Uil1vMtjJ1PbLiPSARHcJh6luXxb7iAUuAURVDEcdsl/SlXt4RPhIEuxR6+yubgtS2iCmbyYF5iKcKutKgMtCv8XI23L+3AW/wTxc/W7zrxbykGmfW15RARkz/hMNyr2SsaycTUMlgBknUdiETCm2wolvZzEmy5OdhSSJD6+4PAJrIHi3nG8QnGm+8J32kQqCDw==
x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW4PR03MB6347.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(53546011)(7696005)(6506007)(30864003)(38070700005)(52536014)(82960400001)(9686003)(55016003)(8936002)(122000001)(5660300002)(71200400001)(508600001)(66446008)(4326008)(66556008)(64756008)(44832011)(76116006)(316002)(86362001)(8676002)(38100700002)(26005)(107886003)(33656002)(66476007)(186003)(83380400001)(2906002)(66946007)(110136005);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?utf-8?B?d3NVTWNsUWpLbUloZkdhNTZyMmszaWh3NmFQWEhJU3RieFhJbTZ6aUNvSlli?=
 =?utf-8?B?emR6V2Nmby94UmFZSUg2N1ppeTJoay9Ta2ZNYzg0aTRuMkJlRTROSk5sWXJR?=
 =?utf-8?B?ZlBFb1k4T2JRbitMZTl2dXAvU24waWIxTlZ0OHh4b0tVS09uUmVXMFQrZmpp?=
 =?utf-8?B?VHpRSEVIdmhBN2xPUDU5ZldYSU1US29zT1hORUVZRjFBRHd0amJBTTJMOXkv?=
 =?utf-8?B?Yzc5cmtycHU0Zm9nb2gxUlh1Q1dRZXdVMVgzVkx3MXpySEJVUTZEV1lsU1Bs?=
 =?utf-8?B?OUpMQ0RMcnJ6djJhZmRQa3h0b2llNSs4UStINmVsMEx6dE5jVStLckoxTnlk?=
 =?utf-8?B?RDRPcHc4dXBtbXh1QTFKSTBUdzdIVnRXT1p4aU13U29BTENacGJsVEFnT3J0?=
 =?utf-8?B?MU1KV284ZEhud2c4Y0dzdUNaK1FvSFNjSXgrTlRCSHAyTGhDQWNHQkgvaTYz?=
 =?utf-8?B?eVJ1SkFzanFybU1sYUE0dGthMmo2NnJ5aWpwVDNvVFJBL1hXWWd0OFhCRFpq?=
 =?utf-8?B?VXJBZHphWWFTdUJhV283N2hmZzFBVnB2YkszelU3N293UHJjdHJ1SDAxVUp4?=
 =?utf-8?B?MFVaclFmd2dSbkhaaEUvTHVvYWg5L3NIVDhua1VIQUhwSVgrS0Rid3pYaEVQ?=
 =?utf-8?B?Tm5sYlJvcDRKTVE0MXk3SXkxNlA5ZU5tM1ViY0U4cW9hWHhvUFdVZHFKY0or?=
 =?utf-8?B?MW9rZ1BIWi90aVg4dUcyQzVITHFZU2FWSW1oMTIzakpVQ05KL3R4QmMxNHo4?=
 =?utf-8?B?VlJ4R1ZkTDI2ODJ6dms2Q00zRHhsZlpRZE51WlJ4MkhtZlhlaGhsV1NZM1Qx?=
 =?utf-8?B?bnQ2YTZ0VUJrYlB6UUhobE5TQmhJbGowMjFtdHltMTdLVkgrTzFWV3BqSjBU?=
 =?utf-8?B?UUZVbWQvTnA2R2FGWlQ2blNRYUV6c21hNzJScE9WdExsazVCdysvM3haSzkw?=
 =?utf-8?B?MjRSdkZpczBLeFBRM2pjbE1jVmpIUEpRQXpzQkovaHg3MHY4R0ZZcUxBb2g0?=
 =?utf-8?B?RTlTWEVIcVdiR0FOK3R0S2luUVowenBqcmMwcnNmcW90K0VFckQxdUJaRlBE?=
 =?utf-8?B?Y0lFWUo3T09IV0FmNFRmVTBZeE1UVWpDUWFtS1A3M3pReVA1N25PV1BVdGV6?=
 =?utf-8?B?T2VGK0l6R00zZmxiaTZEQW1wZTlqQVBXOHBGNU9GOUIvZE5vZ0JScm1TZndR?=
 =?utf-8?B?VyttUm1RWG0wN2g1NURZeU9IT0RPRU05QTVXN1Y1aSs5R1V2SjJJZkIvZndT?=
 =?utf-8?B?Smpaa1EzSlpOZ3NwZlp4WG5wR3hoUGcvdXM1dmdVTnF4ODlTbU9CMXFzM2dC?=
 =?utf-8?B?YjN6VW0ya0VQaHU5RkNSNEg1Z0lRUnQzWDZZcVlrZnh3N0ZFdENNQ3gxWTB1?=
 =?utf-8?B?OTFrclU2bTh6TktMaUpkVjVsQW1HZThrQTk0UVoweU94ZFlId1ovcitkZ0tF?=
 =?utf-8?B?SVlmYWlkcy9JaUdaNXJjcGhrOTExenJZKzBEWGZUYWQvbWd3TW9qSUw4VXEr?=
 =?utf-8?B?bTJXdE9QbmY2OU5wakVQVjZJWi9mcFBOK1F6RVVPa09mbzczUGdGVGVzK2ZI?=
 =?utf-8?B?aUVYOWQ1aElPZXg5L0l1SXZkQzBPK20xWmpIa3dsMDN1ZTRscWZXOGJjNkZB?=
 =?utf-8?B?NFlsT2RKbGY3c1JMV2cxNDcxS0ZqcjRVd2tnZWhOQjlITlo2Q2ZPNnNpNGZr?=
 =?utf-8?B?dUt6N0ptcTFYcG9CUTZjWWtwYWFmcXl2S3FoTEhPeU9lMjY3bkRNeThoMVM5?=
 =?utf-8?B?MGhvR0lmUlhqREE2L2hyRlcyQTFGcmJacGJMSGFDeVl3Tk16cVJvSHFLK1N5?=
 =?utf-8?B?b2RNTHVvblZwREo1K0VjRno4UWsydDJEZ0VWWUhGa04vTXlBaWxjREpaWjRY?=
 =?utf-8?B?YjJvd1RtcXJleFNlSlBJSUUzdDFtNHJiSXlpL2ZQQkNGQmIzeFoyeVZGY0Y5?=
 =?utf-8?Q?6fVk0YmCBb23bz/YCebjDuZVeJZS3wmH?=
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MW4PR03MB6347.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 50c8a9eb-d963-4a62-7500-08da0b54d04d
X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Mar 2022 16:06:52.5773
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 4TdCDjowyP9MldDx0qeD5L56n46m6t9R9Yk/sI6tOv19hi7F9INeR/st/QZaKacI+cGbLqd/7p5HMcuRFUYWwRVv3ya0XcBHHvpLszt//cI=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR03MB3620
X-OriginatorOrg: citrix.com

DQoNCi0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQpGcm9tOiB3aW4tcHYtZGV2ZWwgPHdpbi1w
di1kZXZlbC1ib3VuY2VzQGxpc3RzLnhlbnByb2plY3Qub3JnPiBPbiBCZWhhbGYgT2YgRHVycmFu
dCwgUGF1bA0KU2VudDogMTkgTWFyY2ggMjAyMiAxNzozOQ0KVG86IHdpbi1wdi1kZXZlbEBsaXN0
cy54ZW5wcm9qZWN0Lm9yZw0KU3ViamVjdDogUmU6IFtQQVRDSF0gV2luZG93cyBQViBkcml2ZXJz
IGZhaWwgdG8gc2V0IHVwIFJTUyB3aGVuIHZDUFVzID4gOA0KDQpbQ0FVVElPTiAtIEVYVEVSTkFM
IEVNQUlMXSBETyBOT1QgcmVwbHksIGNsaWNrIGxpbmtzLCBvciBvcGVuIGF0dGFjaG1lbnRzIHVu
bGVzcyB5b3UgaGF2ZSB2ZXJpZmllZCB0aGUgc2VuZGVyIGFuZCBrbm93IHRoZSBjb250ZW50IGlz
IHNhZmUuDQoNCg0KPiBJIHRoaW5rIHdlIG5lZWQgYSBiaXQgbW9yZSBpbiB0aGUgY29tbWl0IGNv
bW1lbnQuIFdoYXQgaXMgdGhlIG5hdHVyZSBvZiB0aGUgZmFpbHVyZS4uLiBhbmQgZG9lcyBYRU5O
RVQgYWR2ZXJ0aXNlIG1vcmUgdGhhbiA4IHF1ZXVlcywgc28gd2lsbCB0aGUgc2l0dWF0aW9uIGV2
ZXIgYXJpc2U/IExpbnV4IGNlcnRhaW5seSB0b3BzIG91dCBhdCA4IHF1ZXVlcy4NCg0KSSBkb24n
dCBiZWxpZXZlIHRoYXQgWGVuTmV0IGV2ZXIgYWR2ZXJ0aXNlcyBtb3JlIHRoYW4gOCBxdWV1ZXMs
IGJ1dCB0aGF0J3Mgbm90IHF1aXRlIHRoZSBzYW1lIGFzIHN1cHBvcnRpbmcgbW9yZSB0aGFuIDgg
dkNQVSdzDQoNClBlcmhhcHMgc29tZXRoaW5nIGluIHRoZSBjb21tZW50IGxpa2U6ICJNYXBwaW5n
IGJldHdlZW4gUXVldWVzIGFuZCBWUFUncyBmYWlscyBmb3IgbW9yZSB0aGFuIDggVkNQVSdzIGJl
Y2F1c2UgdGhlIGJhc2Ugb2YgdGhlIGluZGlyZWN0aW9uIGlzIGFsd2F5cyBjb25zaWRlcmVkIHRv
IGJlIHplcm8sIGFuZCB0aGUgbWFwcGluZyBpcyBhbHdheXMgcGVyZm9ybWVkIG9uIGEgZGlyZWN0
IHZDUFUgbnVtYmVyIGJhc2lzIg0KDQpXb3VsZCB0aGF0IGFkZXF1YXRlbHkgc3VtbWFyaXNlIHRo
ZSBwcm9ibGVtPyAoSXQncyBub3QgZWFzeSB0byBleHBsYWluIHN1Y2NpbmN0bHkgaW4gRW5nbGlz
aCEpDQoNCk1ILg0KDQpUaGlzIHN1bW1hcnkgaGVscGZ1bGx5IHByb3ZpZGVkIGJ5IEVkdmluIFRv
cm9rLg0KDQoNCk9uIGEgVk0gd2l0aCA+OCB2Q1BVcyBSU1MgbWlnaHQgbm90IHdvcmsgYmVjYXVz
ZSB0aGUgZHJpdmVyIGZhaWxzIHRvIHNldCB1cCB0aGUgaW5kaXJlY3Rpb24gdGFibGUuDQoNClRo
aXMgY2F1c2VzIHRoZSBWTSB0byBvbmx5IGJlIGFibGUgdG8gcmVhY2ggMTIuNEdiaXQvcyB3aXRo
ICdpcGVyZjMgLVAgOCcsIGluc3RlYWQgb2YgMTYtMThHYml0L3Mgd2l0aCBhIHdvcmtpbmcgUlNT
IHNldHVwOg0KDQpUaGlzIGNhbiBiZSBlYXNpbHkgcmVwcm9kdWNlZCBpZiB5b3UgZ2l2ZSBhIFZN
IDMyIHZDUFVzIGFuZCBjcmVhdGUgMyBuZXR3b3JrIGludGVyZmFjZXMuIFdpbmRvd3Mgd2lsbCBh
c3NpZ24gMC0zIHRvIG9uZSBuZXR3b3JrIGludGVyZmFjZSwgNC03IHRvIG5leHQsIGFuZCB3aWxs
IHRyeSA4LTEyIEkgdGhpbmsgZm9yIG5leHQgYnV0IHRoZSBkcml2ZXIgcmVqZWN0cyB0aGF0Og0K
DQpQUyBDOlxQcm9ncmFtIEZpbGVzXENJdHJpeFxYZW5Ub29sc1xEaWFnbm9zdGljcz4gR2V0LU5l
dEFkYXB0ZXJSU1MNCg0KDQpOYW1lICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICA6IEV0aGVybmV0IDUNCkludGVyZmFjZURlc2NyaXB0aW9uICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIDogWGVuU2VydmVyIFBWIE5ldHdvcmsgRGV2aWNlICMyDQpFbmFibGVkICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICA6IFRydWUNCk51bWJlck9mUmVj
ZWl2ZVF1ZXVlcyAgICAgICAgICAgICAgICAgICAgICAgICAgIDogOA0KUHJvZmlsZSAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgOiBOVU1BU3RhdGljDQpCYXNlUHJvY2Vz
c29yOiBbR3JvdXA6TnVtYmVyXSAgICAgICAgICAgICAgICAgICA6IDA6MA0KTWF4UHJvY2Vzc29y
OiBbR3JvdXA6TnVtYmVyXSAgICAgICAgICAgICAgICAgICAgOiAwOjMxDQpNYXhQcm9jZXNzb3Jz
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICA6IDQNClJzc1Byb2Nlc3NvckFycmF5
OiBbR3JvdXA6TnVtYmVyL05VTUEgRGlzdGFuY2VdIDogMDowLzAgIDA6MS8wICAwOjIvMCAgMDoz
LzAgIDA6NC8wICAwOjUvMCAgMDo2LzAgIDA6Ny8wDQogICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIDA6OC8wICAwOjkvMCAgMDoxMC8wICAwOjExLzAgIDA6
MTIvMCAgMDoxMy8wICAwOjE0LzAgIDA6MTUvMA0KICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAwOjE2LzAgIDA6MTcvMCAgMDoxOC8wICAwOjE5LzAgIDA6
MjAvMCAgMDoyMS8wICAwOjIyLzAgIDA6MjMvMA0KICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAwOjI0LzAgIDA6MjUvMCAgMDoyNi8wICAwOjI3LzAgIDA6
MjgvMCAgMDoyOS8wICAwOjMwLzAgIDA6MzEvMA0KSW5kaXJlY3Rpb25UYWJsZTogW0dyb3VwOk51
bWJlcl0gICAgICAgICAgICAgICAgOg0KDQpOYW1lICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICA6IEV0aGVybmV0IDQNCkludGVyZmFjZURlc2NyaXB0aW9uICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIDogWGVuU2VydmVyIFBWIE5ldHdvcmsgRGV2aWNlICMxDQpF
bmFibGVkICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICA6IFRydWUNCk51
bWJlck9mUmVjZWl2ZVF1ZXVlcyAgICAgICAgICAgICAgICAgICAgICAgICAgIDogOA0KUHJvZmls
ZSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgOiBOVU1BU3RhdGljDQpC
YXNlUHJvY2Vzc29yOiBbR3JvdXA6TnVtYmVyXSAgICAgICAgICAgICAgICAgICA6IDA6MA0KTWF4
UHJvY2Vzc29yOiBbR3JvdXA6TnVtYmVyXSAgICAgICAgICAgICAgICAgICAgOiAwOjMxDQpNYXhQ
cm9jZXNzb3JzICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICA6IDQNClJzc1Byb2Nl
c3NvckFycmF5OiBbR3JvdXA6TnVtYmVyL05VTUEgRGlzdGFuY2VdIDogMDowLzAgIDA6MS8wICAw
OjIvMCAgMDozLzAgIDA6NC8wICAwOjUvMCAgMDo2LzAgIDA6Ny8wDQogICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDA6OC8wICAwOjkvMCAgMDoxMC8wICAw
OjExLzAgIDA6MTIvMCAgMDoxMy8wICAwOjE0LzAgIDA6MTUvMA0KICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAwOjE2LzAgIDA6MTcvMCAgMDoxOC8wICAw
OjE5LzAgIDA6MjAvMCAgMDoyMS8wICAwOjIyLzAgIDA6MjMvMA0KICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAwOjI0LzAgIDA6MjUvMCAgMDoyNi8wICAw
OjI3LzAgIDA6MjgvMCAgMDoyOS8wICAwOjMwLzAgIDA6MzEvMA0KSW5kaXJlY3Rpb25UYWJsZTog
W0dyb3VwOk51bWJlcl0gICAgICAgICAgICAgICAgOiAwOjQgICAwOjUgICAgIDA6NiAgICAgMDo3
ICAgICAwOjQgICAgIDA6NSAgICAgMDo2ICAgICAwOjcNCiAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgMDo0ICAgMDo1ICAgICAwOjYgICAgIDA6NyAgICAg
MDo0ICAgICAwOjUgICAgIDA6NiAgICAgMDo3DQogICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgIDA6NCAgIDA6NSAgICAgMDo2ICAgICAwOjcgICAgIDA6NCAg
ICAgMDo1ICAgICAwOjYgICAgIDA6Nw0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAwOjQgICAwOjUgICAgIDA6NiAgICAgMDo3ICAgICAwOjQgICAgIDA6
NSAgICAgMDo2ICAgICAwOjcNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgMDo0ICAgMDo1ICAgICAwOjYgICAgIDA6NyAgICAgMDo0ICAgICAwOjUgICAg
IDA6NiAgICAgMDo3DQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIDA6NCAgIDA6NSAgICAgMDo2ICAgICAwOjcgICAgIDA6NCAgICAgMDo1ICAgICAwOjYg
ICAgIDA6Nw0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAwOjQgICAwOjUgICAgIDA6NiAgICAgMDo3ICAgICAwOjQgICAgIDA6NSAgICAgMDo2ICAgICAw
OjcNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgMDo0
ICAgMDo1ICAgICAwOjYgICAgIDA6NyAgICAgMDo0ICAgICAwOjUgICAgIDA6NiAgICAgMDo3DQog
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDA6NCAgIDA6
NSAgICAgMDo2ICAgICAwOjcgICAgIDA6NCAgICAgMDo1ICAgICAwOjYgICAgIDA6Nw0KICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAwOjQgICAwOjUgICAg
IDA6NiAgICAgMDo3ICAgICAwOjQgICAgIDA6NSAgICAgMDo2ICAgICAwOjcNCiAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgMDo0ICAgMDo1ICAgICAwOjYg
ICAgIDA6NyAgICAgMDo0ICAgICAwOjUgICAgIDA6NiAgICAgMDo3DQogICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDA6NCAgIDA6NSAgICAgMDo2ICAgICAw
OjcgICAgIDA6NCAgICAgMDo1ICAgICAwOjYgICAgIDA6Nw0KICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAwOjQgICAwOjUgICAgIDA6NiAgICAgMDo3ICAg
ICAwOjQgICAgIDA6NSAgICAgMDo2ICAgICAwOjcNCiAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgMDo0ICAgMDo1ICAgICAwOjYgICAgIDA6NyAgICAgMDo0
ICAgICAwOjUgICAgIDA6NiAgICAgMDo3DQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIDA6NCAgIDA6NSAgICAgMDo2ICAgICAwOjcgICAgIDA6NCAgICAg
MDo1ICAgICAwOjYgICAgIDA6Nw0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAwOjQgICAwOjUgICAgIDA6NiAgICAgMDo3ICAgICAwOjQgICAgIDA6NSAg
ICAgMDo2ICAgICAwOjcNCg0KTmFtZSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgOiBFdGhlcm5ldCAzDQpJbnRlcmZhY2VEZXNjcmlwdGlvbiAgICAgICAgICAgICAg
ICAgICAgICAgICAgICA6IFhlblNlcnZlciBQViBOZXR3b3JrIERldmljZSAjMA0KRW5hYmxlZCAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgOiBUcnVlDQpOdW1iZXJPZlJl
Y2VpdmVRdWV1ZXMgICAgICAgICAgICAgICAgICAgICAgICAgICA6IDgNClByb2ZpbGUgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDogTlVNQVN0YXRpYw0KQmFzZVByb2Nl
c3NvcjogW0dyb3VwOk51bWJlcl0gICAgICAgICAgICAgICAgICAgOiAwOjANCk1heFByb2Nlc3Nv
cjogW0dyb3VwOk51bWJlcl0gICAgICAgICAgICAgICAgICAgIDogMDozMQ0KTWF4UHJvY2Vzc29y
cyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgOiA0DQpSc3NQcm9jZXNzb3JBcnJh
eTogW0dyb3VwOk51bWJlci9OVU1BIERpc3RhbmNlXSA6IDA6MC8wICAwOjEvMCAgMDoyLzAgIDA6
My8wICAwOjQvMCAgMDo1LzAgIDA6Ni8wICAwOjcvMA0KICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAwOjgvMCAgMDo5LzAgIDA6MTAvMCAgMDoxMS8wICAw
OjEyLzAgIDA6MTMvMCAgMDoxNC8wICAwOjE1LzANCiAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgMDoxNi8wICAwOjE3LzAgIDA6MTgvMCAgMDoxOS8wICAw
OjIwLzAgIDA6MjEvMCAgMDoyMi8wICAwOjIzLzANCiAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgMDoyNC8wICAwOjI1LzAgIDA6MjYvMCAgMDoyNy8wICAw
OjI4LzAgIDA6MjkvMCAgMDozMC8wICAwOjMxLzANCkluZGlyZWN0aW9uVGFibGU6IFtHcm91cDpO
dW1iZXJdICAgICAgICAgICAgICAgIDogMDowICAgMDoxICAgICAwOjIgICAgIDA6MyAgICAgMDow
ICAgICAwOjEgICAgIDA6MiAgICAgMDozDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIDA6MCAgIDA6MSAgICAgMDoyICAgICAwOjMgICAgIDA6MCAgICAg
MDoxICAgICAwOjIgICAgIDA6Mw0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAwOjAgICAwOjEgICAgIDA6MiAgICAgMDozICAgICAwOjAgICAgIDA6MSAg
ICAgMDoyICAgICAwOjMNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgMDowICAgMDoxICAgICAwOjIgICAgIDA6MyAgICAgMDowICAgICAwOjEgICAgIDA6
MiAgICAgMDozDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIDA6MCAgIDA6MSAgICAgMDoyICAgICAwOjMgICAgIDA6MCAgICAgMDoxICAgICAwOjIgICAg
IDA6Mw0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAw
OjAgICAwOjEgICAgIDA6MiAgICAgMDozICAgICAwOjAgICAgIDA6MSAgICAgMDoyICAgICAwOjMN
CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgMDowICAg
MDoxICAgICAwOjIgICAgIDA6MyAgICAgMDowICAgICAwOjEgICAgIDA6MiAgICAgMDozDQogICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDA6MCAgIDA6MSAg
ICAgMDoyICAgICAwOjMgICAgIDA6MCAgICAgMDoxICAgICAwOjIgICAgIDA6Mw0KICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAwOjAgICAwOjEgICAgIDA6
MiAgICAgMDozICAgICAwOjAgICAgIDA6MSAgICAgMDoyICAgICAwOjMNCiAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgMDowICAgMDoxICAgICAwOjIgICAg
IDA6MyAgICAgMDowICAgICAwOjEgICAgIDA6MiAgICAgMDozDQogICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDA6MCAgIDA6MSAgICAgMDoyICAgICAwOjMg
ICAgIDA6MCAgICAgMDoxICAgICAwOjIgICAgIDA6Mw0KICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAwOjAgICAwOjEgICAgIDA6MiAgICAgMDozICAgICAw
OjAgICAgIDA6MSAgICAgMDoyICAgICAwOjMNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgMDowICAgMDoxICAgICAwOjIgICAgIDA6MyAgICAgMDowICAg
ICAwOjEgICAgIDA6MiAgICAgMDozDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIDA6MCAgIDA6MSAgICAgMDoyICAgICAwOjMgICAgIDA6MCAgICAgMDox
ICAgICAwOjIgICAgIDA6Mw0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAwOjAgICAwOjEgICAgIDA6MiAgICAgMDozICAgICAwOjAgICAgIDA6MSAgICAg
MDoyICAgICAwOjMNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgMDowICAgMDoxICAgICAwOjIgICAgIDA6MyAgICAgMDowICAgICAwOjEgICAgIDA6MiAg
ICAgMDozDQpUaGVyZSBpcyBhIGJ1aWx0aW4gaGFyZGNvZGVkIGxpbWl0IG9mIDggcXVldWVzLCB3
aGljaCBpcyBmaW5lLCBidXQgdGhhdCBzaG91bGQgYmUgY29tcGxldGVseSB1bnJlbGF0ZWQgdG8g
Q1BVIG51bWJlcnMhICh0aGUgdG90YWwgbnVtYmVyIG9mIENQVXMgYXNzaWduZWQgdG8gYSBOSUMg
c2hvdWxkIGJlIDw9OCwgc3VyZSkuDQoNClBvdGVudGlhbCBjb2RlIGNhdXNpbmcgaXNzdWUgaW4g
eGVudmlmIHJlY2VpdmVyLmM6DQoNCiBmb3IgKEluZGV4ID0gMDsgSW5kZXggPCBTaXplOyBJbmRl
eCsrKSB7DQogICAgICAgICAgUXVldWVNYXBwaW5nW0luZGV4XSA9IEtlR2V0UHJvY2Vzc29ySW5k
ZXhGcm9tTnVtYmVyKCZQcm9jZXNzb3JNYXBwaW5nW0luZGV4XSk7DQogICAgICAgICAgaWYgKFF1
ZXVlTWFwcGluZ1tJbmRleF0gPj0gTnVtUXVldWVzKQ0KICAgICAgICAgICAgICBnb3RvIGZhaWwy
Ow0KICAgICAgfQ0KDQoodGhlcmUgaXMgYWxzbyBhIHByb2JsZW0gdGhhdCB0aGUgY29kZSBhc3N1
bWVzIHRoYXQgZ3JvdXAgbnVtYmVyIGlzIGFsd2F5cyAwLiBGb3Igbm93IHRoYXQgaXMgdHJ1ZSwg
YnV0IG1pZ2h0IGNoYW5nZSBpZiB3ZSBpbXBsZW1lbnQgdk5VTUEgaW4gdGhlIGZ1dHVyZSkuDQoN
Ckp1biAgMiAwOTo1NDo1NyBwcm9zdCBxZW11LWRtLTQxWzMwODE4XTogMzA4MThAMTYyMjYyNzY5
Ny40NTAyMzM6eGVuX3BsYXRmb3JtX2xvZyB4ZW4gcGxhdGZvcm06IHhlbnZpZnxSZWNlaXZlclVw
ZGF0ZUhhc2hNYXBwaW5nOiBmYWlsMg0KSnVuICAyIDA5OjU0OjU3IHByb3N0IHFlbXUtZG0tNDFb
MzA4MThdOiAzMDgxOEAxNjIyNjI3Njk3LjQ1MDMyMDp4ZW5fcGxhdGZvcm1fbG9nIHhlbiBwbGF0
Zm9ybTogeGVudmlmfFJlY2VpdmVyVXBkYXRlSGFzaE1hcHBpbmc6IGZhaWwxIChjMDAwMDAwZCkN
Ckp1biAgMiAwOTo1NDo1NyBwcm9zdCBxZW11LWRtLTQxWzMwODE4XTogMzA4MThAMTYyMjYyNzY5
Ny40NTIwOTc6eGVuX3BsYXRmb3JtX2xvZyB4ZW4gcGxhdGZvcm06IHhlbnZpZnxSZWNlaXZlclVw
ZGF0ZUhhc2hNYXBwaW5nOiBmYWlsMg0KSnVuICAyIDA5OjU0OjU3IHByb3N0IHFlbXUtZG0tNDFb
MzA4MThdOiAzMDgxOEAxNjIyNjI3Njk3LjQ1MjE4MDp4ZW5fcGxhdGZvcm1fbG9nIHhlbiBwbGF0
Zm9ybTogeGVudmlmfFJlY2VpdmVyVXBkYXRlSGFzaE1hcHBpbmc6IGZhaWwxIChjMDAwMDAwZCkN
Ckp1biAgMiAwOTo1NjoxNCBwcm9zdCBxZW11LWRtLTQxWzMwODE4XTogMzA4MThAMTYyMjYyNzc3
NC4zNzQ3MTM6eGVuX3BsYXRmb3JtX2xvZyB4ZW4gcGxhdGZvcm06IHhlbnZpZnxSZWNlaXZlclVw
ZGF0ZUhhc2hNYXBwaW5nOiBmYWlsMg0KSnVuICAyIDA5OjU2OjE0IHByb3N0IHFlbXUtZG0tNDFb
MzA4MThdOiAzMDgxOEAxNjIyNjI3Nzc0LjM3NDc5ODp4ZW5fcGxhdGZvcm1fbG9nIHhlbiBwbGF0
Zm9ybTogeGVudmlmfFJlY2VpdmVyVXBkYXRlSGFzaE1hcHBpbmc6IGZhaWwxIChjMDAwMDAwZCkN
Ckp1biAgMiAwOTo1NjoxNCBwcm9zdCBxZW11LWRtLTQxWzMwODE4XTogMzA4MThAMTYyMjYyNzc3
NC4zNzcxMjE6eGVuX3BsYXRmb3JtX2xvZyB4ZW4gcGxhdGZvcm06IHhlbnZpZnxSZWNlaXZlclVw
ZGF0ZUhhc2hNYXBwaW5nOiBmYWlsMg0KSnVuICAyIDA5OjU2OjE0IHByb3N0IHFlbXUtZG0tNDFb
MzA4MThdOiAzMDgxOEAxNjIyNjI3Nzc0LjM3NzIwMzp4ZW5fcGxhdGZvcm1fbG9nIHhlbiBwbGF0
Zm9ybTogeGVudmlmfFJlY2VpdmVyVXBkYXRlSGFzaE1hcHBpbmc6IGZhaWwxIChjMDAwMDAwZCkN
Ckp1biAgMiAwOTo1OTowMCBwcm9zdCBxZW11LWRtLTQyWzExMDZdOiAxMTA2QDE2MjI2Mjc5NDAu
NjcyOTQxOnhlbl9wbGF0Zm9ybV9sb2cgeGVuIHBsYXRmb3JtOiB4ZW52aWZ8UmVjZWl2ZXJVcGRh
dGVIYXNoTWFwcGluZzogZmFpbDINCkp1biAgMiAwOTo1OTowMCBwcm9zdCBxZW11LWRtLTQyWzEx
MDZdOiAxMTA2QDE2MjI2Mjc5NDAuNjczMDU4Onhlbl9wbGF0Zm9ybV9sb2cgeGVuIHBsYXRmb3Jt
OiB4ZW52aWZ8UmVjZWl2ZXJVcGRhdGVIYXNoTWFwcGluZzogZmFpbDEgKGMwMDAwMDBkKQ0KSnVu
ICAyIDA5OjU5OjAwIHByb3N0IHFlbXUtZG0tNDJbMTEwNl06IDExMDZAMTYyMjYyNzk0MC42NzU4
OTE6eGVuX3BsYXRmb3JtX2xvZyB4ZW4gcGxhdGZvcm06IHhlbnZpZnxSZWNlaXZlclVwZGF0ZUhh
c2hNYXBwaW5nOiBmYWlsMg0KSnVuICAyIDA5OjU5OjAwIHByb3N0IHFlbXUtZG0tNDJbMTEwNl06
IDExMDZAMTYyMjYyNzk0MC42NzU5OTM6eGVuX3BsYXRmb3JtX2xvZyB4ZW4gcGxhdGZvcm06IHhl
bnZpZnxSZWNlaXZlclVwZGF0ZUhhc2hNYXBwaW5nOiBmYWlsMSAoYzAwMDAwMGQpDQpKdW4gIDIg
MDk6NTk6MzkgcHJvc3QgcWVtdS1kbS00M1s0MDc0XTogNDA3NEAxNjIyNjI3OTc5LjM2Mzg5Mjp4
ZW5fcGxhdGZvcm1fbG9nIHhlbiBwbGF0Zm9ybTogeGVudmlmfFJlY2VpdmVyVXBkYXRlSGFzaE1h
cHBpbmc6IGZhaWwyDQpKdW4gIDIgMDk6NTk6MzkgcHJvc3QgcWVtdS1kbS00M1s0MDc0XTogNDA3
NEAxNjIyNjI3OTc5LjM2NDAwODp4ZW5fcGxhdGZvcm1fbG9nIHhlbiBwbGF0Zm9ybTogeGVudmlm
fFJlY2VpdmVyVXBkYXRlSGFzaE1hcHBpbmc6IGZhaWwxIChjMDAwMDAwZCkNCkp1biAgMiAwOTo1
OTozOSBwcm9zdCBxZW11LWRtLTQzWzQwNzRdOiA0MDc0QDE2MjI2Mjc5NzkuMzY1ODYxOnhlbl9w
bGF0Zm9ybV9sb2cgeGVuIHBsYXRmb3JtOiB4ZW52aWZ8UmVjZWl2ZXJVcGRhdGVIYXNoTWFwcGlu
ZzogZmFpbDINCkp1biAgMiAwOTo1OTozOSBwcm9zdCBxZW11LWRtLTQzWzQwNzRdOiA0MDc0QDE2
MjI2Mjc5NzkuMzY1OTQ5Onhlbl9wbGF0Zm9ybV9sb2cgeGVuIHBsYXRmb3JtOiB4ZW52aWZ8UmVj
ZWl2ZXJVcGRhdGVIYXNoTWFwcGluZzogZmFpbDEgKGMwMDAwMDBkKQ0KSnVuICAyIDEwOjA0OjQx
IHByb3N0IHFlbXUtZG0tNDVbOTcwNV06IDk3MDVAMTYyMjYyODI4MS45MzU4NzE6eGVuX3BsYXRm
b3JtX2xvZyB4ZW4gcGxhdGZvcm06IHhlbnZpZnxSZWNlaXZlclVwZGF0ZUhhc2hNYXBwaW5nOiBm
YWlsMg0KSnVuICAyIDEwOjA0OjQxIHByb3N0IHFlbXUtZG0tNDVbOTcwNV06IDk3MDVAMTYyMjYy
ODI4MS45MzU5NjU6eGVuX3BsYXRmb3JtX2xvZyB4ZW4gcGxhdGZvcm06IHhlbnZpZnxSZWNlaXZl
clVwZGF0ZUhhc2hNYXBwaW5nOiBmYWlsMSAoYzAwMDAwMGQpDQpKdW4gIDIgMTA6MDQ6NDEgcHJv
c3QgcWVtdS1kbS00NVs5NzA1XTogOTcwNUAxNjIyNjI4MjgxLjkzNzg0OTp4ZW5fcGxhdGZvcm1f
bG9nIHhlbiBwbGF0Zm9ybTogeGVudmlmfFJlY2VpdmVyVXBkYXRlSGFzaE1hcHBpbmc6IGZhaWwy
DQpKdW4gIDIgMTA6MDQ6NDEgcHJvc3QgcWVtdS1kbS00NVs5NzA1XTogOTcwNUAxNjIyNjI4Mjgx
LjkzNzkxODp4ZW5fcGxhdGZvcm1fbG9nIHhlbiBwbGF0Zm9ybTogeGVudmlmfFJlY2VpdmVyVXBk
YXRlSGFzaE1hcHBpbmc6IGZhaWwxIChjMDAwMDAwZCkNCkp1biAgMiAxMDowNTowMCBwcm9zdCBx
ZW11LWRtLTQ2WzExNDg0XTogMTE0ODRAMTYyMjYyODMwMC45NzM0ODc6eGVuX3BsYXRmb3JtX2xv
ZyB4ZW4gcGxhdGZvcm06IHhlbnZpZnxSZWNlaXZlclVwZGF0ZUhhc2hNYXBwaW5nOiBmYWlsMg0K
SnVuICAyIDEwOjA1OjAwIHByb3N0IHFlbXUtZG0tNDZbMTE0ODRdOiAxMTQ4NEAxNjIyNjI4MzAw
Ljk3MzU4ODp4ZW5fcGxhdGZvcm1fbG9nIHhlbiBwbGF0Zm9ybTogeGVudmlmfFJlY2VpdmVyVXBk
YXRlSGFzaE1hcHBpbmc6IGZhaWwxIChjMDAwMDAwZCkNCkp1biAgMiAxMDowNTowMCBwcm9zdCBx
ZW11LWRtLTQ2WzExNDg0XTogMTE0ODRAMTYyMjYyODMwMC45NzY1NTQ6eGVuX3BsYXRmb3JtX2xv
ZyB4ZW4gcGxhdGZvcm06IHhlbnZpZnxSZWNlaXZlclVwZGF0ZUhhc2hNYXBwaW5nOiBmYWlsMg0K
SnVuICAyIDEwOjA1OjAwIHByb3N0IHFlbXUtZG0tNDZbMTE0ODRdOiAxMTQ4NEAxNjIyNjI4MzAw
Ljk3NjY1MDp4ZW5fcGxhdGZvcm1fbG9nIHhlbiBwbGF0Zm9ybTogeGVudmlmfFJlY2VpdmVyVXBk
YXRlSGFzaE1hcHBpbmc6IGZhaWwxIChjMDAwMDAwZCkNCkp1biAgMiAxMDoyMjo1NCBwcm9zdCBx
ZW11LWRtLTQ5WzIxOTAxXTogMjE5MDFAMTYyMjYyOTM3NC43MjA3Njk6eGVuX3BsYXRmb3JtX2xv
ZyB4ZW4gcGxhdGZvcm06IHhlbnZpZnxQZG9HZXRJbnRlcmZhY2VHdWlkOiBmYWlsMSAoYzAwMDAw
MzQpDQpKdW4gIDIgMTA6MjI6NTUgcHJvc3QgcWVtdS1kbS00OVsyMTkwMV06IDIxOTAxQDE2MjI2
MjkzNzUuMTk0MTIyOnhlbl9wbGF0Zm9ybV9sb2cgeGVuIHBsYXRmb3JtOiB4ZW52aWZ8UmVjZWl2
ZXJVcGRhdGVIYXNoTWFwcGluZzogZmFpbDINCkp1biAgMiAxMDoyMjo1NSBwcm9zdCBxZW11LWRt
LTQ5WzIxOTAxXTogMjE5MDFAMTYyMjYyOTM3NS4xOTQyMzE6eGVuX3BsYXRmb3JtX2xvZyB4ZW4g
cGxhdGZvcm06IHhlbnZpZnxSZWNlaXZlclVwZGF0ZUhhc2hNYXBwaW5nOiBmYWlsMSAoYzAwMDAw
MGQpDQpKdW4gIDIgMTA6MjI6NTUgcHJvc3QgcWVtdS1kbS00OVsyMTkwMV06IDIxOTAxQDE2MjI2
MjkzNzUuMTk2NzI2Onhlbl9wbGF0Zm9ybV9sb2cgeGVuIHBsYXRmb3JtOiB4ZW52aWZ8UmVjZWl2
ZXJVcGRhdGVIYXNoTWFwcGluZzogZmFpbDINCkp1biAgMiAxMDoyMjo1NSBwcm9zdCBxZW11LWRt
LTQ5WzIxOTAxXTogMjE5MDFAMTYyMjYyOTM3NS4xOTY4MjU6eGVuX3BsYXRmb3JtX2xvZyB4ZW4g
cGxhdGZvcm06IHhlbnZpZnxSZWNlaXZlclVwZGF0ZUhhc2hNYXBwaW5nOiBmYWlsMSAoYzAwMDAw
MGQpDQpKdW4gIDIgMTA6MjM6MzggcHJvc3QgcWVtdS1kbS01MFsyNDUwOV06IDI0NTA5QDE2MjI2
Mjk0MTguNTMwMDQ2Onhlbl9wbGF0Zm9ybV9sb2cgeGVuIHBsYXRmb3JtOiB4ZW52aWZ8UmVjZWl2
ZXJVcGRhdGVIYXNoTWFwcGluZzogZmFpbDINCkp1biAgMiAxMDoyMzozOCBwcm9zdCBxZW11LWRt
LTUwWzI0NTA5XTogMjQ1MDlAMTYyMjYyOTQxOC41MzAxMTU6eGVuX3BsYXRmb3JtX2xvZyB4ZW4g
cGxhdGZvcm06IHhlbnZpZnxSZWNlaXZlclVwZGF0ZUhhc2hNYXBwaW5nOiBmYWlsMSAoYzAwMDAw
MGQpDQpKdW4gIDIgMTA6MjM6MzggcHJvc3QgcWVtdS1kbS01MFsyNDUwOV06IDI0NTA5QDE2MjI2
Mjk0MTguNTMxODExOnhlbl9wbGF0Zm9ybV9sb2cgeGVuIHBsYXRmb3JtOiB4ZW52aWZ8UmVjZWl2
ZXJVcGRhdGVIYXNoTWFwcGluZzogZmFpbDINCkp1biAgMiAxMDoyMzozOCBwcm9zdCBxZW11LWRt
LTUwWzI0NTA5XTogMjQ1MDlAMTYyMjYyOTQxOC41MzE4ODg6eGVuX3BsYXRmb3JtX2xvZyB4ZW4g
cGxhdGZvcm06IHhlbnZpZnxSZWNlaXZlclVwZGF0ZUhhc2hNYXBwaW5nOiBmYWlsMSAoYzAwMDAw
MGQpDQpKdW4gIDIgMTA6MzA6MjggcHJvc3QgcWVtdS1kbS01MVsyODUzMF06IDI4NTMwQDE2MjI2
Mjk4MjguNTEwOTY4Onhlbl9wbGF0Zm9ybV9sb2cgeGVuIHBsYXRmb3JtOiB4ZW52aWZ8UmVjZWl2
ZXJVcGRhdGVIYXNoTWFwcGluZzogZmFpbDINCkp1biAgMiAxMDozMDoyOCBwcm9zdCBxZW11LWRt
LTUxWzI4NTMwXTogMjg1MzBAMTYyMjYyOTgyOC41MTEwNTA6eGVuX3BsYXRmb3JtX2xvZyB4ZW4g
cGxhdGZvcm06IHhlbnZpZnxSZWNlaXZlclVwZGF0ZUhhc2hNYXBwaW5nOiBmYWlsMSAoYzAwMDAw
MGQpDQpKdW4gIDIgMTA6MzA6MjggcHJvc3QgcWVtdS1kbS01MVsyODUzMF06IDI4NTMwQDE2MjI2
Mjk4MjguNTEzNTcwOnhlbl9wbGF0Zm9ybV9sb2cgeGVuIHBsYXRmb3JtOiB4ZW52aWZ8UmVjZWl2
ZXJVcGRhdGVIYXNoTWFwcGluZzogZmFpbDINCkp1biAgMiAxMDozMDoyOCBwcm9zdCBxZW11LWRt
LTUxWzI4NTMwXTogMjg1MzBAMTYyMjYyOTgyOC41MTM2OTE6eGVuX3BsYXRmb3JtX2xvZyB4ZW4g
cGxhdGZvcm06IHhlbnZpZnxSZWNlaXZlclVwZGF0ZUhhc2hNYXBwaW5nOiBmYWlsMSAoYzAwMDAw
MGQpDQpKdW4gIDIgMTA6NDU6NDMgcHJvc3QgcWVtdS1kbS01MlsyODg5XTogMjg4OUAxNjIyNjMw
NzQzLjU3Mzc5MTp4ZW5fcGxhdGZvcm1fbG9nIHhlbiBwbGF0Zm9ybTogeGVudmlmfFJlY2VpdmVy
VXBkYXRlSGFzaE1hcHBpbmc6IGZhaWwyDQpKdW4gIDIgMTA6NDU6NDMgcHJvc3QgcWVtdS1kbS01
MlsyODg5XTogMjg4OUAxNjIyNjMwNzQzLjU3MzkwNDp4ZW5fcGxhdGZvcm1fbG9nIHhlbiBwbGF0
Zm9ybTogeGVudmlmfFJlY2VpdmVyVXBkYXRlSGFzaE1hcHBpbmc6IGZhaWwxIChjMDAwMDAwZCkN
Ckp1biAgMiAxMDo0NTo0MyBwcm9zdCBxZW11LWRtLTUyWzI4ODldOiAyODg5QDE2MjI2MzA3NDMu
NTc2MTg4Onhlbl9wbGF0Zm9ybV9sb2cgeGVuIHBsYXRmb3JtOiB4ZW52aWZ8UmVjZWl2ZXJVcGRh
dGVIYXNoTWFwcGluZzogZmFpbDINCkp1biAgMiAxMDo0NTo0MyBwcm9zdCBxZW11LWRtLTUyWzI4
ODldOiAyODg5QDE2MjI2MzA3NDMuNTc2Mjk4Onhlbl9wbGF0Zm9ybV9sb2cgeGVuIHBsYXRmb3Jt
OiB4ZW52aWZ8UmVjZWl2ZXJVcGRhdGVIYXNoTWFwcGluZzogZmFpbDEgKGMwMDAwMDBkKQ0KDQpJ
IHRlc3RlZCB3aXRoIGJvdGggV2luMTAgYW5kIFdpbmRvd3MgU2VydmVyIDIwMTYsIHdpdGggdmFy
aW91cyBDUFUgdG9wb2xvZ2llcyAoZS5nLiAxMiB2Q1BVcywgYWxsIG9uIG9uZSBzb2NrZXQgc2hv
d3Mgc2FtZSBpc3N1ZSBvbmNlIHdpbmRvd3Mgc3RhcnRzIGFzc2lnbmluZyBDUFVzPjgpDQoNCkEg
d29ya2Fyb3VuZCBpcyB0byBzZXQgTWF4UHJvY2Vzc29yTnVtYmVyIHRvIDcsIHRob3VnaCBvYnZp
b3VzbHkgdGhpcyB3aWxsIGxpbWl0IHNjYWxhYmlsaXR5IHNpbmNlIHZDUFVzID4gOCB3b24ndCBi
ZSB1c2VkIGV2ZW4gaWYgeW91IGhhdmUgbXVsdGlwbGUgVklGczoNCg0KDQoNCg==


From win-pv-devel-bounces@lists.xenproject.org Mon Mar 21 16:19:03 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Mon, 21 Mar 2022 16:19:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.293081.497814 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nWKkJ-0003Sq-0a; Mon, 21 Mar 2022 16:19:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 293081.497814; Mon, 21 Mar 2022 16:19:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nWKkI-0003Sg-TE; Mon, 21 Mar 2022 16:19:02 +0000
Received: by outflank-mailman (input) for mailman id 293081;
 Mon, 21 Mar 2022 16:19:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=pDjm=UA=citrix.com=prvs=0726b29d2=martin.harvey@srs-se1.protection.inumbo.net>)
 id 1nWKkH-0003SZ-FK
 for win-pv-devel@lists.xenproject.org; Mon, 21 Mar 2022 16:19:01 +0000
Received: from esa4.hc3370-68.iphmx.com (esa4.hc3370-68.iphmx.com
 [216.71.155.144]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9c89bae7-a932-11ec-a405-831a346695d4;
 Mon, 21 Mar 2022 17:18:59 +0100 (CET)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9c89bae7-a932-11ec-a405-831a346695d4
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1647879539;
  h=from:to:subject:date:message-id:references:in-reply-to:
   content-transfer-encoding:mime-version;
  bh=C9vFh7eDYDzDmbYhtEIZeYsPxpsf6ZdkEThYUdsyom4=;
  b=h2fV8SxuUqjoGa8ZqaZ6EwqfUM8cs6UXANbfD1qu7D/E5G6QuyZ3s3nJ
   tyS/B5/Cc8OGf0OqCFPbs254WUdpC0NGWgFSBJF5uhQgEnBL8NCxwqIoi
   Mc/iC037EcNCC3SviHLHellXCdwHJE3p+0v7mpvsjs2hrUbsSKTeYzhZR
   k=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
X-SBRS: 5.1
X-MesageID: 69151601
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:JeaWLqvQQknTHPP2nHe4joHpqefnVC9eMUV32f8akzHdYApBsoF/q
 tZmKWGBM/ncNzOneYx+bIvn9UMD6MfTyNJhT1Bt/i43Ri4T+JbJXdiXEBz9bniYRiHhoOOLz
 Cm8hv3odp1coqr0/0/1WlTZhSAgk/nOHNIQMcacUsxLbVYMpBwJ1FQyw4bVvqYy2YLjW1jX4
 IuoyyHiEATNNwBcYzp8B52r8HuDjNyq0N/PlgVjDRzjlAa2e0g9VPrzF4noR5fLatA88tqBb
 /TC1NmEElbxpH/BPD8HfoHTKSXmSpaKVeSHZ+E/t6KK2nCurQRquko32WZ1he66RFxlkvgoo
 Oihu6BcRi8MBIPmnvQmdCBZLAd5fvBAx5PrIX+W5Jn7I03uKxMAwt1rBUAye4YZ5vx2ESdF8
 vlwxDIlN07ZwbjsmfTiF7cq1p9LwMrDZevzvlllxCvFDPBgQZnZXajbzdRZwC0xloZFGvO2i
 88xN2s2MU2eO0Qn1lE/OL9jh7rxiCXDb2N7rWnP+YUqzm/Y9VkkuFTqGIWMIYHbLSlPpW6Hp
 2SD53q8DhwEOdi3zTue7mnqluLJhTn8Wo8ZCPu/7PECqEWa22g7GBAQE1yhrpGEZlWWAowFb
 RZOo2x38PZ0pBfDosTBswOQhUSqlBgMYP1pEtIc70LQzJX4zwaeLz1RJtJeU+AOuMgzTD0s8
 1aGmdL1GDBi2IGopWKhGqS89m3rZ3VMRYMWTWpdFFZevYG/yG0mpkiXJuuPBpJZmTEc9dvY5
 zmR5BYziLwI5SLg//XqpAuX695AS3Wgc+LU2uk1dj/+hu+aTNT8D2BN1bQ8xawcRLt1tnHb4
 BA5dzG2tYji962lmi2XW/kqF7q0/fuDOzC0qQcxQ8Zxrmz0oi76JtE4DNRCyKBBaJtsRNMUS
 BWL5VM5CGF7YhNGkpObk6ruUp93nMAM5PzuV+zOb8omX3SCXFTvwc2aXmbJhzqFuBF1yckXY
 M7HGe7xXSdyIfk2l1KeGrZCuYLHMwhjnAs/s7iglE/5uVdfDVbIIYo43KymNblosvnb/F2Lr
 76y9aKikn1ibQE3WQGOmaY7JlEWN3krQ5fwrs1cbOmYJQR6XmomDpfsLXkJIuSJQ4w9ej/0w
 0yA
IronPort-HdrOrdr: A9a23:98r8HqgNAeriXP73zRhJmq7gh3BQX2N13DAbv31ZSRFFG/FwyP
 rAoB1L73PJYWgqNU3IwerwQpVoMkmsiKKdhrNhcItKPTOW8ldASbsN0WKM+UyHJ8STzJ8/6U
 4kSdkENDSSNyk3sS+Z2njALz9I+rDum8rI5ds2jU0dNT2CA5sQqTuRYTzrd3GeMTM2YqbRY6
 DsgvavyQDQG0g/X4CePD0oTuLDr9rEmNbNehgdHSMq7wGIkHeB9KP6OwLw5GZQbxp/hZMZtU
 TVmQ3w4auu99uhzAXH6mPV55NK3PP819p4AtCWgMR9EESjtu/oXvUhZ1SxhkF3nAid0idprD
 AKmWZkAy1H0QKQQohym2qo5+Cv6kdo15ao8y7lvZKqm72HeNt9MbsZuWsRSGqm12Mw+N57y6
 5FxGSfqt5eCg7Bhj3045zSWwhtjVfcmwtqrQc/tQ0pbWIlUs4mkWXfxjImLL4QWCbhrIw3Gu
 hnC8/RoP5QbFOBdnjc+m1i2salUHg/FgqPBhFqgL3f7xFG2HRii0cIzs0WmXkNsJo7Vplf/u
 zBdqBljqtHQMMaZb90QO0BXcy0AGrQRg+kChPbHX33UKUcf37doZ/+57s4oOmsZZwT1ZM33I
 /MVVtJ3FRCD34Gyff+qaGj3iq9MFlVBw6du/22z6IJyYHBeA==
X-IronPort-AV: E=Sophos;i="5.90,199,1643691600"; 
   d="scan'208";a="69151601"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=IO86fPWx80l8R7ImNNV3/Q7bE2yHvLqPI+To5ImHMNZBcVKZowCHJ8uqVA0hxwQVNTXTAF1rAapBBS8KzVw6xUhos5fePLJNwrFO48E6R68fBy+WOPSZS+VO9HUbV+QSgRh54nc3KcLB402aLODCBYuB1RngbCtTykvVJLB39lbbE4/AVLTAh4OBBqnJZ4mBtXKsxtL9GMBxGegpZGeLaH9DxSs7qb7vgaPSM1GZ2mZ57eerE60MS92gGYr1Mha9vYjefX00Q8Dw1uMpS1I6Bh0aLxS2hOzSWkI+s5aGL+hOH3Xwb2Sx+i/SXO2Xa0LnX5Zn4bpXAGxp32w5iUZDrw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=C9vFh7eDYDzDmbYhtEIZeYsPxpsf6ZdkEThYUdsyom4=;
 b=gXmbr3BdTsq05/gSN/eRLA1/XPmNINtWvdkIIPy+6Cbm5OgoFWBgRtsPH6zgzVCmVKp9giAA3+lUXb7Y+wNwvbISQ+v0HTWzgvkeaAOEAMbaS2JAco2wt8aSetUdEA35XpUD4hCooPM/tD4G2SY4HA3hrPZWrEePQ2UTBI4y+VA83e80baYgYH58IpGrBtc8Mdr0TmvijM9pj+6EHdFCUdulVVBviQeMc0pPHSTvcvAfhPixWhAqKssnnfUt9TUJMhWGMXscgLn9g9TzSwjjmZ4kPUPnpaMzi4jzvLD5JEIJVKtryOYH+qKxwWVgCRk3dVVjM3Q7zhVwZ+c+g69DTg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=C9vFh7eDYDzDmbYhtEIZeYsPxpsf6ZdkEThYUdsyom4=;
 b=PuWmgYwLa80J7b4Q4W+kyCqc3B8D8TZZJ16ubzKhWYvOsqRdZB4SXo+OwB1wS9/LZRvSUsVkialvAF9mFpZ/Sd1380y5CgHPBOonCANlU9WLqIwlA0cxhrz6RKcL+PBNsOAybLQeZyNTs7R5maeWVWFPJ83PiEuOzI+pafM4zK8=
From: Martin Harvey <martin.harvey@citrix.com>
To: "paul@xen.org" <paul@xen.org>, "win-pv-devel@lists.xenproject.org"
	<win-pv-devel@lists.xenproject.org>
Subject: RE: [PATCH] Fix div by zero in xenvif if no queues.
Thread-Topic: [PATCH] Fix div by zero in xenvif if no queues.
Thread-Index: AQHYOHVErur6M9Oay0m6lfwVxs2ql6zG/w8AgAMM6lA=
Date: Mon, 21 Mar 2022 16:18:52 +0000
Message-ID: <MW4PR03MB6347833BCE009D5F5926C8A29A169@MW4PR03MB6347.namprd03.prod.outlook.com>
References: <20220315140100.1868-1-Martin.Harvey@citrix.com>
 <4dc2d35b-9d73-746e-1ab6-b18d5f75cc66@gmail.com>
In-Reply-To: <4dc2d35b-9d73-746e-1ab6-b18d5f75cc66@gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: d1b3d0c1-9142-48d9-cc68-08da0b567d5f
x-ms-traffictypediagnostic: BYAPR03MB4823:EE_
x-microsoft-antispam-prvs: <BYAPR03MB482339D94C035C32FB901CCE9A169@BYAPR03MB4823.namprd03.prod.outlook.com>
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: PmtFK8hzvX6728+YddlJEgxDt2qESPVTtQ39bDWHUSItJ+qbiGMg0zCipRcarZOlhJO5BDCVfHJNavrBeRNtr7eGiaH9zH51TvNr7qyGTpQHy392H1KX7W4uxwAn+CjAaD2DFI5e4snCyPiTa843JLty7r5ndcNreBeZxx0CFtF9kARPceMXQFFlIuXiTrPPbRwllS+P8uEhaIp+G0KaLgy6KOkHTwEQkJU9EDW62yHrsN1+Uqvj/0wXQ7I8cqIMcghcJ/L4Z/M3DCFq6rDfxOGDvv4QeclkwWC4L5t80hzuhEmq8Ie6uzg6hPwQ2gWKQe0wbaanL274NlWJPxLSE5c5ugr5BPJ8sQ1o+gRUmWlspuM64PdjblhPXOjO6Xy2FXh8Wa0pik7tQ4kzLAcZVvkC4qt3CRfzR8K4Ukfc8IdKHvjC3zJuxoyBK6k+hjfYDK8XpDEN/oEFAP6/eHdUg8+g0BFycy60Vh6OTv5aT31PNamCb8IZj5WTfFSMMpiO4gjKQROROKpeB5ZWtMeFVFHK0mqOSOVe8889yumw+e24aUKg4MrvV/fhB+xgMRXG6pjuHmZ6F8c1gsKiLXM4EQTz9uqfUhTgi+WsSWTowU1pwJbDTS277ZZEJ5WZbVRcr5bu8zt5tA7k5bOCy14HvcO4ttQmNaXmz4P9LpM/THny0PAx7T88OweSc5N4yxblQp+7aOyFVlLHVB+0V5kwiQ==
x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW4PR03MB6347.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(55016003)(9686003)(38070700005)(44832011)(38100700002)(5660300002)(82960400001)(26005)(6506007)(7696005)(508600001)(33656002)(316002)(2906002)(110136005)(122000001)(71200400001)(86362001)(66556008)(66446008)(64756008)(76116006)(66476007)(66946007)(8936002)(186003)(52536014)(83380400001)(8676002);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?utf-8?B?NUNjWGRUZTVBTUpRRFpnRmwzZFo0Q3Y4RTlXLzNGamlNVEE1L0RCRnd0VlZP?=
 =?utf-8?B?UTBOT2FDbk5aYThzbWFLaVh1R1N5bUx6QWdYU2pLRUFZaGtsUmQ4emZGQ0Rm?=
 =?utf-8?B?c2lXaUxuZFdscVN6WFk1WDY5SjIvUVFxV2xydHpQVlY2dnNXV3g3K2tUaFRr?=
 =?utf-8?B?SlRtb0hSNENXKzNmYXo0OXpYV2FBdFpURTh5Zy80bk5aZnFDc1hZem9wVExL?=
 =?utf-8?B?UVF6RHRwVmZaTFNUdTh4b2RpcEUrWHBWekIxaVB2eEEzVVVXOVRrdjhYRVR1?=
 =?utf-8?B?blpmVDFrNTNZeXZ0Y05DSWs0bHoxcGszRUZPZ1dzT3B1VWJOTjZyYmNlbUJT?=
 =?utf-8?B?OEZaMUFiSi9SUHZsUWVPNDNWR3JUMWNlT2ZFZktpZGJzY3FnWW1lWEVQQzBs?=
 =?utf-8?B?SHBEdnBXSnd1c0hFdFNKQjBPUFFjYTZ5TXJWR1YxdkFROENpMXAyTkEzQVo4?=
 =?utf-8?B?NGxiQnNTWGNjdSt2bjFLUXZpdk9YSVV6aUpmTjBrSExzeFc0UlFsbnpQUmxQ?=
 =?utf-8?B?Rk1SSFpJUytrRXNrRjl2NHlJLzl0L0dvb055VzdGTHJ2d29aOWxlNkNBdlhx?=
 =?utf-8?B?MDg1OXVqQkFzNHdhOXJPN3B2VTVGU081YkZzSzJwUTlyajlRY21udW5kL0Ro?=
 =?utf-8?B?dHQwZjU2K3VNTk5lbUxiNURHSHE4Zm44WkV1REpPYnVhUjBOeEVmK3dJRjNw?=
 =?utf-8?B?TGhXalQxai9KbkwrT0dvZDB2eU1aVGVuTXNPb2NqNFE4SFVXdk1tajlCUlJK?=
 =?utf-8?B?SjVuZUN5MzNmK1BaVGlRS2lTM1ZMNVJNbUtFM0Z2WGRwaWxTNCtyeDArVFNI?=
 =?utf-8?B?VmJnTUhuR1g3K3lzZ2RUZDlRN0tWV01leUlwMkdKQmkzdlFHVHpST2JrQWdP?=
 =?utf-8?B?M3JxbWJPbjZoY1VPaDZGOEdJQXpHemtpRmZxRmM3TVVkVno1TzV5WWdFY2w1?=
 =?utf-8?B?dEo4RFhYWERRVGpvOXBTaHhpckJKbktqaXk5UUNPL1hIdU5WL1ZqbkFGRERY?=
 =?utf-8?B?VU5INkY1cTNZT1BXS2I0cHJGVktOWVpVL1ZNMGhrbkdMdTJYOEFYNkU4RWtB?=
 =?utf-8?B?SWpNTGZEeURsbjBlYVRoUjEzdkMyMnhSc3p5ZWxMb0VNWW96dHkxTngwNzlP?=
 =?utf-8?B?Zi83VGVpLy9qcXlnR2gvVTBoZWdIeFdvQ2cwOUJzeHdabWFLM0FiaGxxMFJ6?=
 =?utf-8?B?cVdzeE85ay9mYVV1aHFWQ2tkOFMwYWhlWmUrcUYxMTZXTTJFZ3JQQnRuTVl3?=
 =?utf-8?B?NDVVMEp6WUROWXJIZk1sR0htUjZlYk1xR29Ib3E2MUZTc1lPQWNLZCtReCsy?=
 =?utf-8?B?SXJaQ3pVVVh1WVhOa0Z3ekJ1aC8zRjNIcVIxT3ZkZkVXRkxHaDVSZGtZeFhW?=
 =?utf-8?B?ZUJsVzQ5WVdhUGgvNGUxSTA2TzMrcmg1YXV2RDJZTEhEVkdKSjR6QmZ0QlAz?=
 =?utf-8?B?STcwZ2RCNVVHRHhRUldySG1pWHJCelJZWXB3UGlxVzh3Znk1dnBwSk5hYzYx?=
 =?utf-8?B?K2gwWW5aajduNmFGNnYyU1AxWGtHOWE0bG5oeXd4NWRUblBEa0plUVdjZTZR?=
 =?utf-8?B?TkdBZ2ozRDl4N3BGVS9QZ1orRTVUSlVzNjZhd3R2bXZWdXlLTk1BaW14TWpL?=
 =?utf-8?B?Z1NSUHpidlZNTkFtdDlsTU9NUmE2TFd0L1VuOHY0Sk4rTytod2lBZGc2elpn?=
 =?utf-8?B?ZE1wdHhYWXlCb0FtWkNJV0FEdy9GYkN2OFN2SEpDN2xMVjJpeTg0ZjVsSXVT?=
 =?utf-8?B?eXJrMWttekpFWDExZXRpUHAwODllcUNxMCtkbkJBeGJGVGZyRE9wN3VlbWVN?=
 =?utf-8?B?RWNQV3QyOGl1TWI2ZUxpNnExR0NpT0k3SE9oL2VnK0dtZVFqdEZUZGx5VXZK?=
 =?utf-8?B?YkFUYWliazRJOHJrS1ZKaGwvM3pUQVcrUGFqMUVxKzByY1czVysxREZlNlR2?=
 =?utf-8?Q?U5u91WU9AsHWFBf2W7auHTpDQU9fO4ee?=
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MW4PR03MB6347.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d1b3d0c1-9142-48d9-cc68-08da0b567d5f
X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Mar 2022 16:18:52.3920
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 3HTozjhdie8Ou8529XaSj/BUgYRBWg1tf3q9D310qt928M2Rk/yr3hODawNZCGZiAKHvfZ8HrIcd8mJYX0jQNLf+otB5XFvpXYvhLgAXI4M=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR03MB4823
X-OriginatorOrg: citrix.com

DQoNCi0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQpGcm9tOiB3aW4tcHYtZGV2ZWwgPHdpbi1w
di1kZXZlbC1ib3VuY2VzQGxpc3RzLnhlbnByb2plY3Qub3JnPiBPbiBCZWhhbGYgT2YgRHVycmFu
dCwgUGF1bA0KDQo+IFRoZXJlIGlzIGxvY2tpbmcgdGhhdCBpcyBzdXBwb3NlZCB0byBzdG9wIHRo
YXQgZnJvbSBoYXBwZW5pbmcuIA0KPiBWaWZUcmFuc21pdHRlclF1ZXVlUGFja2V0VmVyc2lvbigp
IGNhbGxzIEFjcXVpcmVNcnN3TG9ja1NoYXJlZCgpIHRvIG1ha2Ugc3VyZSB0aGF0IHN0YXRlIGlz
IG5vdCBjaGFuZ2luZyB1bmRlcm5lYXRoIGl0Lg0KDQpBaCwgeWVzLCBJIHNlZSB3aGF0IHlvdSBt
ZWFuLiBUaGUgdmFyaW91cyBmcm9udGVuZCBzdGF0ZSBjaGFuZ2VzIGFyZSBtZWFudCB0byBiZSBz
eW5jaHJvbmlzZWQgdW5kZXIgdGhlIFZJRiBsb2NrLiAoSG93ZXZlciBJIG5vdGUgdGhhdCB3ZSBt
YXkgbm90IGJlIHRoZSBvbmx5IFZJRiAuLi4uKS4gQW5kIHRoZW4gdGhlcmUncyBhIGxpdHRsZSBi
aXQgb2YgZXh0cmEgc3luYyB0byBkbyB3aXRoIHRoZSBzdXNwZW5kIC8gcmVzdW1lIGVhcmx5L2xh
dGUgY2FsbGJhY2tzLg0KDQo+IEl0IHJlYWxseSBzaG91bGQgbm90IGhhcHBlbiBhdCBhbGwuDQoN
Ck9LLiBJJ2xsIGxvb2sgYmFjayB0aHJvdWdoIHRoZSBsb2dzIGEgYml0IG1vcmUgY2FyZWZ1bGx5
LiBJIHN1c3BlY3QgdGhhdCBhZnRlciByZXN1bWluZyBmcm9tIHN1c3BlbmQsIHRoZSBmcm9udGVu
ZCBzdGF0ZSBpcyBnZXR0aW5nIHNldCB0byBVTktOT1dOIG91dHNpZGUgdGhlIFZJRiBNcnN3IGxv
Y2suIFBvc3NpYmxlIGNhbmRpZGF0ZXMgSSBzaGFsbCB0YWtlIGEgbG9vayBhdDoNCg0KMS4gVklG
IG1yc3cgbG9jayBub3QgY29ycmVjdGx5IHNoYXJlZCBiZXR3ZWVuIGFsbCB1c2VycyBvZiBmcm9u
dGVuZCBzdGF0ZS4NCjIuIEFueSBmcm9udGVuZCBzdGF0ZSBjaGFuZ2VzIGNvbW1hbmRlZCBieSBW
QkQ/IChOb3QgbG9va2VkIHlldCkuDQozLiBTb21lIGVycm9yIHBhdGggd2hpY2ggYnlwYXNzZXMg
dGhlIFZJRiBsb2NrLg0KNC4gU29tZSBjb3JuZXIgY2FzZSAvIHJhY2UgY29uZGl0aW9uIHdpdGgg
ZWFybHkvbGF0ZSBzdXNwZW5kIGNhbGxiYWNrcy4NCjUuIENoYW5nZXMgdG8gdGhlIGZyb250ZW5k
IHN0YXRlIGNvbW1hbmRlZCBieSBvdGhlciBwb3dlciBjYWxsYmFja3Mgd2hpY2ggaGF2ZSBieXBh
c3NlZCB0aGUgVklGIGxvY2suDQoNClRob3NlIGFyZSBhbGwgdGhlIHBvdGVudGlhbCBob2xlcyBJ
IGNhbiB0aGluayBvZiBvZmYgdGhlIHRvcCBvZiBteSBoZWFkIC4uLg0KDQpNSC4NCg==


From win-pv-devel-bounces@lists.xenproject.org Mon Mar 21 16:45:48 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Mon, 21 Mar 2022 16:45:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.293103.497860 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nWLAB-000087-TX; Mon, 21 Mar 2022 16:45:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 293103.497860; Mon, 21 Mar 2022 16:45:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nWLAB-000080-Qh; Mon, 21 Mar 2022 16:45:47 +0000
Received: by outflank-mailman (input) for mailman id 293103;
 Mon, 21 Mar 2022 16:45:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=pDjm=UA=citrix.com=prvs=0726b29d2=martin.harvey@srs-se1.protection.inumbo.net>)
 id 1nWLAA-00007u-EH
 for win-pv-devel@lists.xenproject.org; Mon, 21 Mar 2022 16:45:46 +0000
Received: from esa2.hc3370-68.iphmx.com (esa2.hc3370-68.iphmx.com
 [216.71.145.153]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 596e3b16-a936-11ec-8fbc-03012f2f19d4;
 Mon, 21 Mar 2022 17:45:45 +0100 (CET)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 596e3b16-a936-11ec-8fbc-03012f2f19d4
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1647881144;
  h=from:to:subject:date:message-id:references:in-reply-to:
   content-transfer-encoding:mime-version;
  bh=F4alCZBJuj8d647rUjVEyU2RTvnKjVVjKBrCY8LD6Hw=;
  b=DjcIZoRF56OSD4XBI3a+bXGzI8yiIvV0gvirXKUBhcYPPOGGjwLy0edT
   t3m9gu4YCubedIgVDKCvqu65AbqEThU6utv6azjzkoc5gak0ULc14QfFS
   DdJCn22tW+BBY6NFA6zemP7cL9zfFk3gm5GbmRhZeRNht8PH+60aX9eTm
   g=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
X-SBRS: 5.1
X-MesageID: 66790126
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:BbZD2K6SwkKgE+NJRNxQ1gxRtEHHchMFZxGqfqrLsTDasY5as4F+v
 mdOXWzSM/yONmWhett3YI7jp0ME7JDcnIRjHgM4/i9gHi5G8cbLO4+Ufxz6V8+wwmwvb67FA
 +E2MISowBUcFyeEzvuVGuG96yE6j8lkf5KkYAL+EnkZqTRMFWFw0XqPp8Zj2tQy2YPjWlvX0
 T/Pi5a31GGNimYc3l08s8pvmDs31BglkGpF1rCWTakjUG72zxH5PrpGTU2CByKQrr1vNvy7X
 47+IISRpQs1yfuP5uSNyd4XemVSKlLb0JPnZnB+A8BOiTAazsA+PzpS2FPxpi67hh3Q9+2dx
 umhurSOalYQEZ/+vN1MdAhDHDl4GqhWo6TudC3XXcy7lyUqclPpyvRqSko3IZcZ6qB8BmQmG
 f4wcW5XKErZ3qTvnez9GrIEascLdaEHOKsas2t8zDefDf88X53Sa67L+cVZzHE7gcUm8fP2O
 ZRGOGI2NkqojxtnClstIbEgvO2UiDraeBNKt3zSvYwW7D2GpOB2+Oe0a4eEEjCQfu1Kn0Ddv
 nyD82nnDxUyMN2E1SHD4n+qnvXIny7wRMQVDrLQ3uZjnVm71mEVThoMWjOGTeKR0xDkHYgFc
 gpNp3Ro/fNaGFGXosfVYT7pmVG2jDolCv1BCOIi7iaI2vHPylPMboQbdQJpZNsjvc4wYDUl0
 F6Vgt/kbQBSXK2ppWG1renN827rUcQBBSpbPHJfE1NZizX2iNtr5i8jWOqPB0JcYjfdPTjri
 w6HoyEl71n4pZ5ajv7rlbwrbt/Fm3QocuLXzliMNo5GxlkgDGJAW2BPwQKHhcus1K7DEjG8U
 IEswqByFtwmA5CXjzCqS+4QBryv7PvtGGSC3QE/RsF4rmT0oC/LkWVsDNZWfhoBDyr5UWWxP
 B+7Vf15uve/w0dGnYcoOtnsWqzGPIDrFMj/V+C8Uza9SsMZSeNzxwk3PRT49zm0yCAEyPhjU
 b/GIZfEJStLUsxPkWvpL9rxJJd2n0jSM0uIHsulp/lmuJLDDEOopUAtawLfMLpmsPva/G04M
 b93bqO39vmWa8WnCgH/+o8PN1Eaa389AJH9sctMceCfZAFhHQkc5zX5mNvNp6QNc3xpq9r1
IronPort-HdrOrdr: A9a23:gVKjvqk6xxYBymso2M63duN8MQrpDfIo3DAbv31ZSRFFG/Fw8P
 re+8jztCWE7Ar5PUtKpTnuAsW9qB/nmqKdgrNwAV7BZmfbUQKTRekJgLcKqAeAJwTOssJbyK
 d8Y+xfJbTLfD1HZB/BkWqF+gAbsbu6zJw=
X-IronPort-AV: E=Sophos;i="5.90,199,1643691600"; 
   d="scan'208";a="66790126"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=bl97DUh0xPuzldJ6WluQawT2e8c134E1oQMImUrEGJKng2cfLJw7wwC1HLc3/NzlUsU7frVUVRf5TqwpAshnQZDUCY8+hhIqo+CY6CknsRxdcNcF3VKx+46b6DxVPdKhzWewxy3+ww2Go4o+XaLvcFXGwBWBew8GeARUPu6kWvdEh8kkCQpcaIX4Gxu+u0wPVlOmPxI851tKhtqDScv1Nw0d+eBGceVQv5PgwYFBe9Ap2Nzhz+nmwgYi/hjkBO2SCABRXl8xWHHDEMwH/a/ruyaf63NUdlNXbNb2V6ja5TIY4GKZ6qTa8SyYDAT2nK+fPASn5kA31/pahIn4Vn/T7g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=F4alCZBJuj8d647rUjVEyU2RTvnKjVVjKBrCY8LD6Hw=;
 b=VRi7nlIsUBj6uP2svDFVt5uXSoLtIVdR23vLT6bDCgSb8LF6cA+028P73T3xLy2ifjvYGue4jb2QngdnKag5KJv135q9Wd5hFZ1y9fGxDW9TyrULB5QYW3Win9XqkSuSjnp5kS85wBwtm7DfgaZKSybZ1inQTy7c/aFTRbHge057SHFINDgGSUGu1aKiLIVJdPF7KB1UjR+NqMZya0EXFniVQ0KT3e53CQiwdUOXOa6Jf+bP+OjbaX+YnFQ00aKMNpfDAxTPjiMjBvdDkOhTw+tf48VOK+beN4lOd18/37Vtf80euheAWvqk8bDCemHrC0vBVLAbcR+YezXwymz5Lg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=F4alCZBJuj8d647rUjVEyU2RTvnKjVVjKBrCY8LD6Hw=;
 b=ifU+Vcyohh5TGqIpcFsxvCUboVfEc/IcgSZYHzvtpW+S7ZGuHJehSvzzWusK0R7EIrsNdrsXOA4rLIaF+mZ5WWfCgsioC3eI3xtH+dPLsNcZXnInTatcosvLEebima9AiGQ2OhPmbeLZxu8UYLIVz87mkV8Ba8+fDZ2dRBOuG9w=
From: Martin Harvey <martin.harvey@citrix.com>
To: Martin Harvey <martin.harvey@citrix.com>, "paul@xen.org" <paul@xen.org>,
	"win-pv-devel@lists.xenproject.org" <win-pv-devel@lists.xenproject.org>
Subject: RE: [PATCH] Fix div by zero in xenvif if no queues.
Thread-Topic: [PATCH] Fix div by zero in xenvif if no queues.
Thread-Index: AQHYOHVErur6M9Oay0m6lfwVxs2ql6zG/w8AgAMM6lCAAAji0A==
Date: Mon, 21 Mar 2022 16:45:39 +0000
Message-ID: <MW4PR03MB6347EA178647386BE6EB9E439A169@MW4PR03MB6347.namprd03.prod.outlook.com>
References: <20220315140100.1868-1-Martin.Harvey@citrix.com>
 <4dc2d35b-9d73-746e-1ab6-b18d5f75cc66@gmail.com>
 <MW4PR03MB6347833BCE009D5F5926C8A29A169@MW4PR03MB6347.namprd03.prod.outlook.com>
In-Reply-To: <MW4PR03MB6347833BCE009D5F5926C8A29A169@MW4PR03MB6347.namprd03.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: d451d662-300c-4978-1a5d-08da0b5a3b1e
x-ms-traffictypediagnostic: CY4PR03MB2982:EE_
x-microsoft-antispam-prvs: <CY4PR03MB29827C43649C7C539BD9EF009A169@CY4PR03MB2982.namprd03.prod.outlook.com>
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 6ftMCUDUa1b+s6FT8tXKQ0rjgbhasLBj4Y4eEtmDgTH+C5ayP8ggFewhzQg606AULKCg486hRRO+kYpJ2OW3MwI3drSR5rn4hGNNB86OX1fo8bL1qgVaa75YDeuVtkfXk7FuOXNXPAwETNtO/gDOBah4iY/EcLMtq1+XtKGOMVdSBc/9lNT9cOMgf78uNJaY3g6zEbCR0DpGgnBrRNr9jhRxnQUOfOF0jcP2WXjV+9SkK7B8Wza5F/5IUy2y41k39xcXVRQ6Fv1yojAfMa9yUAxVQPPbk28W7ypsxaISWZoyCx2hEQU58eVfxBlXZ5Btt5SJiF0GuuxL3SW8V95z/eYkuWCgDmpLAclZQWfdZysopmHYtyu+wGA5XfHMXWjsJ7xG53UuvtesRL5GXNQSmW7m05Dn7gboZ0fMESA1eiSysiRrC0QnJ8fzqD3rMd42G+jHIed28gE6k64asQDWwfofMUvi07XU6vWA3hTiSToJjBi5XJ4g5pT6XBu6ffbCgxUDtgsxNDjPq2YRUPGWsltW97OfFrChFDxV8MzuPDHPZSvkAp3BFozfF64ypmNVktSUR3FL4Md6EXp41EKQH/N/euLfbGHDVSLjuwO01FOEU0eHlu06YvAkvNL1S4xM9iHm1zdp9zIfGsg37dyTqmU/AtbCQy2Uo5KrYu4gzl8ckpys1Y7/Ok5W5Gpj4yFT+lFgFJiEufWjEvOeyNuX1Q==
x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW4PR03MB6347.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(9686003)(33656002)(316002)(2940100002)(44832011)(38100700002)(52536014)(86362001)(38070700005)(76116006)(66946007)(122000001)(508600001)(8676002)(66446008)(64756008)(186003)(26005)(82960400001)(6506007)(2906002)(110136005)(7696005)(83380400001)(55016003)(66476007)(5660300002)(71200400001)(66556008)(4744005)(8936002);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?utf-8?B?VEJ0SXN1OGkxb2N0K01aTmR5a2xiVkVhR1Rva0JZMGdza2VtWkc3eGQ3MWhq?=
 =?utf-8?B?VUtnRm1LMnNYZVVkcnUwZHlZMHgzNnl1dXdjOGtpQjVPQnJnRGlaT2dhVDZm?=
 =?utf-8?B?bzdycXh0a0pORkpOdy8wV0s5YnBxSFFvWUpRL2QyeFZkVmpvK05iRTFaaHhr?=
 =?utf-8?B?VHNBbFRpSEVWUjhWK05wNEd0WThwaS9WQS83ZGZpTC95VHVhK0pVRktHcEpZ?=
 =?utf-8?B?VlZ6L2FtZzJpSkZMYkIwcVZBTU9TNG9NeWx4Z2UrR3JnaDRRZjZINzhhdnpX?=
 =?utf-8?B?K2hOLzVtSmxXWDF6YUJQYWhBNkEyYnhnZUxUQ0pXVDRlSUt1TjZqOHJJZFc0?=
 =?utf-8?B?QjNJdTI1NnpTQTgzREI2cEZuWmUzK09PbDdVMUlnU3M2a2h1cVN2T2J3UXVR?=
 =?utf-8?B?QWxMOEhVQ09DeHBvR3h2RG5IREJEeGkwNDdoRDBYVHpTNTFLMzRrOUNjNmt5?=
 =?utf-8?B?Qlo1QWxZRXVzTTdienI1U2Q1WVNnLzQwQ2FwQk1yc2RpT3ZSL25Wa3huUG1p?=
 =?utf-8?B?Z0wxdVFyaHNNMVZzeXdud1hNUG1IbGFFL3dIb29ZRTd6bW5BVWp5Z1BQcmdw?=
 =?utf-8?B?MnAxVEFuemFxZHdRTURjSUFCSHAyQkREOGJkZXIzT0Z5KzROakNsKzZSajQ2?=
 =?utf-8?B?d1pPdStBVld0aHl6L21KRzkzelo2U1Y0VzlSdDlzV2FpZUxBT3dhVDZFQmEy?=
 =?utf-8?B?U3RCZGxDMkVkSm9sK2poOFJxaHIybVRWNVAzQ0FobXBuL3RkK3UxeWlrckpn?=
 =?utf-8?B?c2hQanF2Nnc0cHFiODNLUU5VVThDbWMrQ0VHOW45U0xld2M4NnlxV1E4NEcr?=
 =?utf-8?B?c3ROTkdlNHdIZUJvUkN5bWI0ZjVEbTFkSnVTSEZwVys2aXFrY2VQbFY5ZVZP?=
 =?utf-8?B?eTJvTmc0MlhBdno2eVA2d0I0c1NpbUdjMmQvc1lPWUt5ekxmVzc2V25MUzYv?=
 =?utf-8?B?aHNneGJodmVzQXgzMXpQY3l2T3NDekFScDJmQzFPODhBaFNHVmxjaUpWTDds?=
 =?utf-8?B?bFBqQ1o3Q05YRWdJN3VEdXhXeE4wNDlTcEY3VkR5bGh4MXVBS2JmOW1QVjdN?=
 =?utf-8?B?bzYxSEFYSXJZWkl0d2ZoV0pETXZPeWF6c3BNNURwM3pOU1BXcE9IS0g2Znl3?=
 =?utf-8?B?R0pnbjlzSTdML3BFdUo1bSs5VHE1ZlpldGtEUy9lU25rd0c2MzVRYWNUSTdG?=
 =?utf-8?B?ZGIyOWhuOXAvRVBLSVZrbW15aEZZZWtQRFRpK3FiWjRsaE5TSUg0S1VMNzFm?=
 =?utf-8?B?clBkQisyM3dScWg3Z094YVh3SlBUdCs3WFg5T0VnZm1DMVd2UDJwN2JTQmJY?=
 =?utf-8?B?RWZ5a3ZqSHNsMEZCYVNqd3pWMk5lWGxmNWN4WVFrYUF4MEVKOWVPKzQ4WGlK?=
 =?utf-8?B?Z1d5N1lUNW1ERFpxbXVza2lDcnl0Y05EdTZONkErS2tMWWIvSndGcGkxaGxu?=
 =?utf-8?B?V28wbkVLdERnRU1PUXdtL3pvZ3UzMTJXZFNaVVlKZkdKMFdhRnlob0l6dUt1?=
 =?utf-8?B?eW1oQklaamxOalNQcmtPNS9YWUNSRzcyQ09oeFlaNDhtY0ZGclJ0dVVvNE00?=
 =?utf-8?B?VjBraXpLbGxJeXphRDlvUS82WisyS0w2QUxQMWQxRGpIcjNNUjhyMkJ2UDk4?=
 =?utf-8?B?aCtBZFMyVDVWaHNheUVSU2ZMZ1hQY0wwSEZFUTE4bllTMi9MbkxnQmZveWlD?=
 =?utf-8?B?RUlnOXN3QU1ZRkxwU1lST3FjRjF6aUxXU1J1ZVlLSngvNlI1d2NmQ0lBa3ZX?=
 =?utf-8?B?SldiYWVZUUM2STJ0TFdFQVRtd3ZSdWw3Uzh6Mm5nMHZFRzFPNEg0RUg1cWV0?=
 =?utf-8?B?dWNnL2hLSGFGc2Z5YWZ6bHFNOTlZRkVFaDU0aFgyeWJaT2d0Z20rTVhHTEhy?=
 =?utf-8?B?aGZGMjE4a0hCeFpBYTJNNmNEcWtienR4NmV4VzNUZTRUM3hjRXlQSm9zdDIx?=
 =?utf-8?Q?4iBUbs/w6U5iFO+MpgiSYeDYdzj3bwcs?=
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MW4PR03MB6347.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d451d662-300c-4978-1a5d-08da0b5a3b1e
X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Mar 2022 16:45:39.2199
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: gqMmg8/XGHlKgRngVNCjd0qZ+hpjdRX6aDVSuRpb00S/W3vgamwZSAeLsZquOqgSY7EjF1iYxY4papTQencLLvMYrp7HJb9k0FweG3+ZrMw=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR03MB2982
X-OriginatorOrg: citrix.com

DQotLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KRnJvbTogd2luLXB2LWRldmVsIDx3aW4tcHYt
ZGV2ZWwtYm91bmNlc0BsaXN0cy54ZW5wcm9qZWN0Lm9yZz4gT24gQmVoYWxmIE9mIER1cnJhbnQs
IFBhdWwNCg0KPiBJdCByZWFsbHkgc2hvdWxkIG5vdCBoYXBwZW4gYXQgYWxsLg0KDQpJIGFsc28g
bm90aWNlLCB3aXRoIHNvbWUgdHJlcGlkYXRpb24sIHRoYXQgRnJvbnRlbmRTZXROdW1RdWV1ZXMg
ZG9lczoNCg0KICAgIHN0YXR1cyA9IFhFTkJVU19TVE9SRShSZWFkLA0KICAgICAgICAgICAgICAg
ICAgICAgICAgICAmRnJvbnRlbmQtPlN0b3JlSW50ZXJmYWNlLA0KICAgICAgICAgICAgICAgICAg
ICAgICAgICBOVUxMLA0KICAgICAgICAgICAgICAgICAgICAgICAgICBfX0Zyb250ZW5kR2V0QmFj
a2VuZFBhdGgoRnJvbnRlbmQpLA0KICAgICAgICAgICAgICAgICAgICAgICAgICAibXVsdGktcXVl
dWUtbWF4LXF1ZXVlcyIsDQogICAgICAgICAgICAgICAgICAgICAgICAgICZCdWZmZXIpOw0KICAg
IGlmIChOVF9TVUNDRVNTKHN0YXR1cykpIHsNCiAgICAgICAgQmFja2VuZE1heFF1ZXVlcyA9IChV
TE9ORylzdHJ0b3VsKEJ1ZmZlciwgTlVMTCwgMTApOw0KDQpTbyBpbiB0aGUgY2FzZSB3aGVyZSB0
aGUgY29udGVudHMgb2Ygb3VyIHhlbnN0b3JlIGJ1ZmZlciBpcyBub3QgYW4gaW50ZWdlciwgd2Un
bGwgc2V0IGJhY2tlbmQgbWF4IHF1ZXVlcyAoYW5kIGhlbmNlIGZyb250ZW5kIG1heCBxdWV1ZXMp
IHRvIHplcm8sIHdpdGhvdXQgc28gbXVjaCBvZiBhIHdoaW1wZXIgb2YgYW4gZXJyb3IsIHNvIHRo
YXQgcHJvYiBuZWVkcyB0byBiZSBzb3J0ZWQgdG9vLg0KDQpNSC4NCg==


From win-pv-devel-bounces@lists.xenproject.org Mon Mar 21 17:15:38 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Mon, 21 Mar 2022 17:15:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.293144.497893 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nWLd3-0004gs-Hh; Mon, 21 Mar 2022 17:15:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 293144.497893; Mon, 21 Mar 2022 17:15:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nWLd3-0004gl-Eq; Mon, 21 Mar 2022 17:15:37 +0000
Received: by outflank-mailman (input) for mailman id 293144;
 Mon, 21 Mar 2022 17:15:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ot5V=UA=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1nWLd1-0004gf-H7
 for win-pv-devel@lists.xenproject.org; Mon, 21 Mar 2022 17:15:35 +0000
Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com
 [2a00:1450:4864:20::32d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 84d7f709-a93a-11ec-8fbc-03012f2f19d4;
 Mon, 21 Mar 2022 18:15:34 +0100 (CET)
Received: by mail-wm1-x32d.google.com with SMTP id
 k124-20020a1ca182000000b0038c9cf6e2a6so2670865wme.0
 for <win-pv-devel@lists.xenproject.org>; Mon, 21 Mar 2022 10:15:34 -0700 (PDT)
Received: from ?IPV6:2a00:23c5:5785:9a01:ad9a:ab78:5748:a7ec?
 ([2a00:23c5:5785:9a01:ad9a:ab78:5748:a7ec])
 by smtp.gmail.com with ESMTPSA id
 c5-20020a5d63c5000000b002040822b680sm7861331wrw.81.2022.03.21.10.15.33
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 21 Mar 2022 10:15:33 -0700 (PDT)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 84d7f709-a93a-11ec-8fbc-03012f2f19d4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=message-id:date:mime-version:user-agent:reply-to:subject
         :content-language:to:references:from:in-reply-to
         :content-transfer-encoding;
        bh=2OnYf3zcSCY+ZOCkRCLoueAbvgPemI8KHO26NdTOipo=;
        b=Y5oOQI83teLnvqz3ChkvXBP+NqHsgawvyIBcRw+/9nK2qLPilVwhjaR5IwxSIDDVRG
         uN0Nk/lp/STq7JJNrjM42ScVfBV5YIVqXeAyvNHgt0TyaltUB5jXTl/KjV6cWQt55mB+
         f0JO0F0PIea2hAD7rRiwgCrFYL0A6m4qs7/gjujLnOC2EewIxkIKh39ehgqrwJcmP+hc
         DHqTxTy49EcFx4rKm8RV87Z/l3zq/0LJ95gqlJ8WDeOLlkAnTvtNd2glMdrwIN6Y97cf
         3/DMz/xF11+8elms8YKEuoucCPWe7hd8Ga4uaX29PYskfYrDW5g+BUTApy7HI0RJnRSy
         0WHQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:message-id:date:mime-version:user-agent:reply-to
         :subject:content-language:to:references:from:in-reply-to
         :content-transfer-encoding;
        bh=2OnYf3zcSCY+ZOCkRCLoueAbvgPemI8KHO26NdTOipo=;
        b=iUzNDZ059XIv/q2ECOBfIceUPlb3I4pN9QkbXMRayAF4lMU0GaRXfzE4wPjiNlFnW1
         b4PRglwZ8ZI/6nPj4CRJOF2KguKN3+KPrNZwAx8Nkqsie6L4VA4i+pvKpyWpizfo/dkd
         YLbQQp9T8Ex7ZNqMBf1QMU8ppSKN8p98qTAa1TQBOsbmwKslcD+/oYymzxGtlo4J0PYf
         cMWrt3C6sLm+LqDcZNZRaln3uXhDg4Ya/FBg3dlh5TALe3AWcJVqOUxZEtlDTAkRdzgB
         zGn7saYvIsfEslZ4lRw7urxQ8oBchvdbYk2GU37NArr6XD/a2xUPUufHBbr/90IjgoQQ
         Oezw==
X-Gm-Message-State: AOAM532uwIz4LpgfNcoHUjNhnp0aVAAJGyqflWgbspTMbaFE7KOybNt8
	TLm5pe9BdRk0RaJufCFSYe4=
X-Google-Smtp-Source: ABdhPJz+CisOugfDDiXEuSEMiOntenr4JxDCaZG4GH87mcayr8zRdhHMp03zIw1f0RXo6o1/lHZlfw==
X-Received: by 2002:a05:600c:4fcd:b0:38c:7495:e644 with SMTP id o13-20020a05600c4fcd00b0038c7495e644mr106484wmq.102.1647882933951;
        Mon, 21 Mar 2022 10:15:33 -0700 (PDT)
Message-ID: <67c02d2e-66ca-292a-13ca-a64a16f5a3c0@gmail.com>
Date: Mon, 21 Mar 2022 17:15:32 +0000
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.7.0
Reply-To: paul@xen.org
Subject: Re: [PATCH] Fix div by zero in xenvif if no queues.
Content-Language: en-US
To: Martin Harvey <martin.harvey@citrix.com>, "paul@xen.org" <paul@xen.org>,
 "win-pv-devel@lists.xenproject.org" <win-pv-devel@lists.xenproject.org>
References: <20220315140100.1868-1-Martin.Harvey@citrix.com>
 <4dc2d35b-9d73-746e-1ab6-b18d5f75cc66@gmail.com>
 <MW4PR03MB6347833BCE009D5F5926C8A29A169@MW4PR03MB6347.namprd03.prod.outlook.com>
 <MW4PR03MB6347EA178647386BE6EB9E439A169@MW4PR03MB6347.namprd03.prod.outlook.com>
From: "Durrant, Paul" <xadimgnik@gmail.com>
In-Reply-To: <MW4PR03MB6347EA178647386BE6EB9E439A169@MW4PR03MB6347.namprd03.prod.outlook.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 21/03/2022 16:45, Martin Harvey wrote:
> 
> -----Original Message-----
> From: win-pv-devel <win-pv-devel-bounces@lists.xenproject.org> On Behalf Of Durrant, Paul
> 
>> It really should not happen at all.
> 
> I also notice, with some trepidation, that FrontendSetNumQueues does:
> 
>      status = XENBUS_STORE(Read,
>                            &Frontend->StoreInterface,
>                            NULL,
>                            __FrontendGetBackendPath(Frontend),
>                            "multi-queue-max-queues",
>                            &Buffer);
>      if (NT_SUCCESS(status)) {
>          BackendMaxQueues = (ULONG)strtoul(Buffer, NULL, 10);
> 
> So in the case where the contents of our xenstore buffer is not an integer, we'll set backend max queues (and hence frontend max queues) to zero, without so much of a whimper of an error, so that prob needs to be sorted too.
>

I really wouldn't go down that rathole. If the frontend cannot 
implicitly trust its backend (or the toolstack that creates it) then all 
bets are off IMO.

   Paul


> MH.



From win-pv-devel-bounces@lists.xenproject.org Mon Mar 21 17:23:30 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Mon, 21 Mar 2022 17:23:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.293156.497908 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nWLkf-0006EX-EE; Mon, 21 Mar 2022 17:23:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 293156.497908; Mon, 21 Mar 2022 17:23:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nWLkf-0006EQ-BJ; Mon, 21 Mar 2022 17:23:29 +0000
Received: by outflank-mailman (input) for mailman id 293156;
 Mon, 21 Mar 2022 17:23:28 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=pDjm=UA=citrix.com=prvs=0726b29d2=martin.harvey@srs-se1.protection.inumbo.net>)
 id 1nWLke-0006EG-Bq
 for win-pv-devel@lists.xenproject.org; Mon, 21 Mar 2022 17:23:28 +0000
Received: from esa2.hc3370-68.iphmx.com (esa2.hc3370-68.iphmx.com
 [216.71.145.153]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9cbbccce-a93b-11ec-a405-831a346695d4;
 Mon, 21 Mar 2022 18:23:26 +0100 (CET)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9cbbccce-a93b-11ec-a405-831a346695d4
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1647883406;
  h=from:to:subject:date:message-id:references:in-reply-to:
   content-transfer-encoding:mime-version;
  bh=YuYL1s1V3XkkGJcNqhCkO7Xfynff14yo9E6xqmU2kKs=;
  b=RPhaIg0eVb+vAMvf1TNfZxCa8RCNcEZ6c+OuvolAik8sTRdPDHgbPa5U
   +BrrAiaN3z6DGd25vHmJlpH6X25N8Kb41xt8+uDEH/iMv1UpMt1vgS4fX
   x8S8yYYzC/1RWs4Eu41GyOtdSBV4LaFvNATNbGxt7ae6IS9ISD8U69O9k
   8=;
Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
X-SBRS: 5.1
X-MesageID: 66793697
X-Ironport-Server: esa2.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:BDMZHa4GkYPJ9rjqFYpnpwxRtEHHchMFZxGqfqrLsTDasY5as4F+v
 jBMUDuAa/nfa2CnL9x1Ydi1/EkAuMDWmodkTFRl/igwHi5G8cbLO4+Ufxz6V8+wwmwvb67FA
 +E2MISowBUcFyeEzvuVGuG96yE6j8lkf5KkYAL+EnkZqTRMFWFw0XqPp8Zj2tQy2YPjWlvX0
 T/Pi5a31GGNimYc3l08s8pvmDs31BglkGpF1rCWTakjUG72zxH5PrpGTU2CByKQrr1vNvy7X
 47+IISRpQs1yfuP5uSNyd4XemVSKlLb0JPnZnB+A8BOiTAazsA+PzpS2FPxpi67hh3Q9+2dx
 umhurS6bhlzfafogt8iaBRVMAsnLPN635HIdC3XXcy7lyUqclPpyvRqSko3IZcZ6qB8BmQmG
 f4wcW5XKErZ3qTvnez9GrIEascLdaEHOKsas2t8zDefDf88X53Sa67L+cVZzHE7gcUm8fP2O
 ZRGOGExPUuojxtnZ29JNa0fp96SvV6veH5E+WiVh7Nw2j2GpOB2+Oe0a4eEEjCQfu1Kn0Ddv
 nyD82nnDxUyMN2E1SHD4n+qnvXIny7wRMQVDrLQ3uZjnVm71mEVThoMWjOGTeKR0xDkHYgFc
 gpNp3Ro/fNaGFGXosfVVTqm+j2OmgQnZOFdHs4QsRDT9o367FPMboQbdQJpZNsjvc4wYDUl0
 F6Vgt/kbQBSXK2ppWG1renN827rUcQBBSpbPHJfE1NZizX2iNtr5i8jWOqPB0JcYjfdPTjri
 w6HoyEl71n4pZ5ajv7rlbwrbt/Fm3QocuLXzliMNo5GxlkgDGJAW2BPwQKHhcus1K7DEjG8U
 IEswqByFtwmA5CXjzCqS+4QBryv7PvtGGSC3QE/RsF4rmT0oC/LkWVsDNZWfhoBDyr5UWWxP
 B+7Vf15uve/w0dGnYcoOtnsWqzGPIDrFMj/V+C8Uza9SsMZSeNzxwk3PRT49zm0yCAEyPhjU
 b/GIZfEJStLUsxPkWvpL9rxJJd2n0jSM0uIHsulp/lmuJLDDEOopUAtawLfMLpmsPva/G04M
 b93bqO39vmWa8WnCgH/+o8PN1Eaa389AJH9sctMceCfZAFhHQkc5zX5m9vNp6QNc3xpq9r1
IronPort-HdrOrdr: A9a23:sRyWkqxN4LohJpUrwgBvKrPwA71zdoMgy1knxilNoERuA6ilf8
 DHppsmPGzP+VEssRAb6Kq90ca7MBDhHPJOjLX5Xo3SITUO2lHYTr2KhLGKq1aLJ8S9zJ856U
 4KScZD4ajLbWSS+vya3ODXKbodKZK8gcaVbK/lvg5QpRcDUdAZ0+5WMHfhLnFL
X-IronPort-AV: E=Sophos;i="5.90,199,1643691600"; 
   d="scan'208";a="66793697"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=en+BefJyw1KN3QZIcMxRKCKOq0u8kFn0HVl+kvCyNZSRqlG96KVHfE2KXVHQTTBr+PMA44ASfFM70yEeivwuBUniB9bQ7CmUChdIjVDez5slsyg0mlIFdXHG8ZUo7EATxG9enN7RebbRGryvbDkA3UMvsejetLSrq+MKhYeoBAO+wShT7nIQ1AhvBrrs9LGMYsLZjH1W3KH9CDJN5HYbIaFXJDLlfI/U+tj4i3gr+R6Tv0sO+67nn0SZSTMYltWd/WU2pfc0GAPeXjSvOnT6YG2yclfAv+5ygWODbT22DrWVBMZFUQffw20NLY1Nzp84UWdz/O9n1+bkCVuCV2hkoQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=YuYL1s1V3XkkGJcNqhCkO7Xfynff14yo9E6xqmU2kKs=;
 b=ExT1AQjpk/O+U32AnPlRzhA3d5YTGv3OQ21U0y4tTn+zLi/WWP5Akb9rRs8/OHRbY20RzHZ6SyPP3jAVyMPpiEyfWqtCFEVjUNZKFfDN4MYa6ylir642wyBtp4quTltyPKNyLX6hgeVm1mwcFbaf/5hLTLC/Y1JCszjFaMcOqqgf6jzRazeSUnXopWlDbwu13ZiFiSZLz8e9bWQqRfhO/gkEfsXl1hJGAfWMEl48HUTFuxdIvOrFmOoqQKInKn+c2PI3ieBJtCTcNSbxh1+svYl1nUPUnw3T45w2zur+N37a9AAUTVfTbbch6lrs+ltq5HHlsQEjPBUzkEcOzLPxrA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=YuYL1s1V3XkkGJcNqhCkO7Xfynff14yo9E6xqmU2kKs=;
 b=SsptrIGrs+3DGNSVVfQG+hctXWOshDlWDJgx2FeOXoWuCwPJTQTcK6CrXhbCu5v0QuPBlC7U/bQxMmBa5sy0/m0phTJRNSA0HXYuGgegkTYzAjfwrqbzEhdXWbBFYMHC53ZePnsZg70mQXXU3nCm3Q1LF6+MvIjcBKjfKtRh6m0=
From: Martin Harvey <martin.harvey@citrix.com>
To: "paul@xen.org" <paul@xen.org>, "win-pv-devel@lists.xenproject.org"
	<win-pv-devel@lists.xenproject.org>
Subject: RE: [PATCH] Fix div by zero in xenvif if no queues.
Thread-Topic: [PATCH] Fix div by zero in xenvif if no queues.
Thread-Index: AQHYOHVErur6M9Oay0m6lfwVxs2ql6zG/w8AgAMM6lCAAAji0IAACMQAgAABpIA=
Date: Mon, 21 Mar 2022 17:23:19 +0000
Message-ID: <MW4PR03MB63475551B99F57E123DB7F489A169@MW4PR03MB6347.namprd03.prod.outlook.com>
References: <20220315140100.1868-1-Martin.Harvey@citrix.com>
 <4dc2d35b-9d73-746e-1ab6-b18d5f75cc66@gmail.com>
 <MW4PR03MB6347833BCE009D5F5926C8A29A169@MW4PR03MB6347.namprd03.prod.outlook.com>
 <MW4PR03MB6347EA178647386BE6EB9E439A169@MW4PR03MB6347.namprd03.prod.outlook.com>
 <67c02d2e-66ca-292a-13ca-a64a16f5a3c0@gmail.com>
In-Reply-To: <67c02d2e-66ca-292a-13ca-a64a16f5a3c0@gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: ac7873d4-2aae-4bf8-3bf5-08da0b5f7e8c
x-ms-traffictypediagnostic: BY5PR03MB5268:EE_
x-microsoft-antispam-prvs: <BY5PR03MB5268808F7967D4350EBE67E69A169@BY5PR03MB5268.namprd03.prod.outlook.com>
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 9Go1MuIoZBhZ6KP3rpA6TWp4VCIAPsffpsyJjAy5PGw5vXn0FzhDhckYyIf/SDjkJlUiluIGvY8ZAYTYM0EDWXVW5G1QySi46et+RTJDuViA4xg4EjJLpCxI+K/NHjH0t+dzoYkBnsrhCqJJzucMtMjBDcx2bGcbqpiItsSMBSmRAv2Ov0/nlb7KyLJpWRRSJR/ZjMjYF8fyjemmSdnhidwwBmdeTRJpI+6KFazmrQrjWadPRLDjzL5LIg0/1ucbUO4TxlwaMckBe0TTGCRbt9TDaybCHRQYbRAwuTCBJOXdhAtFZYM7xkXBhqxkVqKpUJVqNeM47Yudb+0BIHv6YCpYSqNYDgLhtgjKRilaZs4EODQMpgejLZHNs7NMzMaJdwxAnzspfqEEyFrPc8jB+/iUez5DXiIGPMQoNqywhFo9JdMuR6LyicigB/+vD/E7qkLlvyzoSUH+FO8JdVMnUTu/rM4suaC5dDpXfECkfCkHFe5S9aRxQ4v03qVUGnQ2tyhW5wWT95U3qV3i+3Sn2MSg+qLkPLptDAgziQ9o+NWionGyQJBhVCl20ZvzQ1AZnc9AaQssXt+H+3LrB/0iqPix/9nubT6RDjqIG1LB/GMIrFA89Q0rRjxOiXMhaiMMEPoiLbbvF7YMWwPxY83PlppaYi5M53XlHpCuuRGLot6+cNV+8xAdCZieNzigqxWn7N6ECmf/xbGnfXiZIhoB5g==
x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW4PR03MB6347.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(82960400001)(26005)(186003)(122000001)(38100700002)(9686003)(83380400001)(5660300002)(44832011)(8936002)(4744005)(52536014)(86362001)(55016003)(33656002)(508600001)(2906002)(71200400001)(6506007)(7696005)(76116006)(38070700005)(8676002)(64756008)(66446008)(66476007)(66556008)(66946007)(110136005)(316002);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?utf-8?B?bVdmZDRYRGpFK1JCMXZMM1NUMnY2VUg3WkRxQ2pIbDR5WlM1ZGx2TnNUZzR0?=
 =?utf-8?B?Zkwra2J0VUxXY3NDR0t4alFOTGdZVzZIWWU4VHZ0RWxydE5qWThQcEpLQ28v?=
 =?utf-8?B?SUFQUGpBSFdNQ3Rzc29wK0l0dDVFVU5IM1Q3ZzhwZWZXcWJubU1qdWVFZUda?=
 =?utf-8?B?YjFRQ0dZZnhVOUZGblFsK0wyQWhoUjNINytveHdKTm5Ea2hmRG9hRFJPeE5D?=
 =?utf-8?B?RjU2K0hmM1Iyd0RacHQyRjA4UU9sSFRTVW53L1REVllOUFg1MlR3bnlUS1Mv?=
 =?utf-8?B?TWlXd21hZjFDYVF0QlJVc3RxSGRqVWR6Tkt4N0ZtYmM4b2poYU1PZ0J4Sm5o?=
 =?utf-8?B?aFB0elA3dmJ4a2FuR0l5OU42VVVUbG5pVTJJeGZjUCt3elRrMkllTThJZU9w?=
 =?utf-8?B?SUx2UTQ0bjEyYisvdmdSdkMvMUk4U1RtOUdEbEt5NWo4WlpjSVl1SnpDTW1Y?=
 =?utf-8?B?NUNSRXR0YXZUZnFFNkc2aXdUdFZOZVQ1dzVtem1vdVdnR3ZyUDVSQjVoV1Yz?=
 =?utf-8?B?VytsSFdWQ2hNQU9XaHZwTkZKUHQzQ05WS0xSbHFrS3BhRHk4VlVkZGdRWERZ?=
 =?utf-8?B?VXV5bTl2R2pIQ3FRa1VFNGR6L1haZnV1enpBQmk2TDl3UTMzTG1kNC9hU291?=
 =?utf-8?B?TFViMHZoZlQ0UmlGT0VwN3dQWm1sVTB4MFZucFdnc0NpWVpUQ2IxTjQ0Wlcz?=
 =?utf-8?B?SVBxMmMzeVhsKzU1THhRUGROYkJPRUVJS29rZHhJNmNWOUNXWk9RQnBGcjFS?=
 =?utf-8?B?UlhPS01IWWVueVczQXc2L21lV21iSVNXc3c3cDN5cElxL1lOYnpSL3U1V1pG?=
 =?utf-8?B?eXcrTyttNUFaV3hqL01aK2dGM0FPdVR2NzkwS0xBQkdLK2NFWndKR3dRQXJr?=
 =?utf-8?B?ZjBydjljeFBLZ0dTeC95ZFhVWVhlem13UGpBRjFmbnV4d1A4WXA2aEZBeVND?=
 =?utf-8?B?ZmZaWXVHSFlTcFVrRG1FdUNGM0Z4QzZLZjhEWWpnYlVOaUtwMEU3V3dSaTRR?=
 =?utf-8?B?QUZ5U1dxSmtUVjRWQ0FFMjQ2VjArWU1LZjBQMjVkNUxyRDlHdlJrUWJFeWZk?=
 =?utf-8?B?c1NqVW5OeVU3TkhINTZ0R0M1N2lMaGYyU1FvcHNUeHd3T3JsZjRKR3FJR3p3?=
 =?utf-8?B?OVN1aTZBd0tHSm54dWZVWW83VUtZaVQzL1I1Q0p2bnpGZXhxMll0T3NLa0s3?=
 =?utf-8?B?TFZ5OTZDQ1RWc1RYSml3MG85QU5oQU5JMW1adERDWnhqRDdVaTNxZEZCQ0Ey?=
 =?utf-8?B?VXQxZ0poNDJOMitUekViZ2VxRVRWS3ZGak51akRRMWJVQUZNSDRrUVZIV0dF?=
 =?utf-8?B?SXVScEM2WTlPdkdjdjdmWVVwc0UxS2lndmFldVN5Ri9wSlIyRTlEWkI0enRH?=
 =?utf-8?B?R2JoeWlMSW5VRXFZSjYyWUJTY3JjRm1rdElXUUErYllpUEc0UGhRQlNYczJZ?=
 =?utf-8?B?N1NaSmp0bWJvWU9CMFJwWCs4a3dkWVFmVEFNTW1MTzg0amMvN1ZOdXVhMFVX?=
 =?utf-8?B?Y3RSUkR2ZmRjZ0dTUmRHVkNRTmZ5SlFBZjVSMHc1dkNnZm5nbjJjNnZnckw1?=
 =?utf-8?B?RXF5UHE3Yk9iY3ZITnNHVWFsNGdudHB6MXZ0T1VldFBBdFBlZnZsMlA1NkVq?=
 =?utf-8?B?NmFESk1UMURma1pHUTlyTEVIbVB2WjZhVDUzNkVmRS8xM1k0S1FVOXJQRWRz?=
 =?utf-8?B?VFhBRkZUdUZUc1hCSU9TRDRoZmptY0dwVmVnY0daODhLUGdhajFQcmE3aHkw?=
 =?utf-8?B?R3VmS3dIYXE1RUk1eEhTSTdydXk3K2M3MDVwQlcxTWNvNDhyYUZ5ZFBmL1pV?=
 =?utf-8?B?dFI5TmxFN2dCd3o2OXl0QjljNnVOQTlRd0hyeE5kcW8rb0JYbDlzYWdpR3FV?=
 =?utf-8?B?U2w0VXZWNko0a2NQamQvWmxrM1dMUDJDVTVENXNWSk9uRkZvRnpBUEtJcVRF?=
 =?utf-8?Q?QSOgys29cicYnzsP+G8UaVp3gtQDVLJD?=
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MW4PR03MB6347.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ac7873d4-2aae-4bf8-3bf5-08da0b5f7e8c
X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Mar 2022 17:23:19.8171
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: pADCTDxiOqegSqvyqPd8Wsxz3xiM+9G2zkvqgmxzKk3wTa9w57NBPA0HAlJQ9tvoYlC/7VxGPX93DNkLGPZYHbMmXzNR3uPvHtXKrBtddFY=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR03MB5268
X-OriginatorOrg: citrix.com

DQoNCi0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQpGcm9tOiBEdXJyYW50LCBQYXVsIDx4YWRp
bWduaWtAZ21haWwuY29tPiANCg0KDQo+IEkgcmVhbGx5IHdvdWxkbid0IGdvIGRvd24gdGhhdCBy
YXRob2xlLiBJZiB0aGUgZnJvbnRlbmQgY2Fubm90IGltcGxpY2l0bHkgdHJ1c3QgaXRzIGJhY2tl
bmQgKG9yIHRoZSB0b29sc3RhY2sgdGhhdCBjcmVhdGVzIGl0KSB0aGVuIGFsbCBiZXRzIGFyZSBv
ZmYgSU1PLg0KDQpXZWxsLCB3ZSBtYXkgeWV0IGhhdmUgdG8gZm9yIHRoZSBzYWtlIG9mIHNvZnR3
YXJlIHJvYnVzdG5lc3MuIEknbGwgcmUtbG9vayB0aHJvdWdoIHRoZSBsb2dzIGFuZCB0aGUgdmFy
aW91cyBwb3NzaWJsZSBlcnJvciBjYXNlcywgYW5kIGdldCBiYWNrIHRvIHlvdSBvbiB0aGlzIG9u
ZS4gSSB0aGluayB0aGlzIGZhbGxzIHVuZGVyIHRoZSBjYXRlZ29yeSBvZiAiZGVmZW5zaXZlIHBy
b2dyYW1taW5nIi4gSXQgd291bGQgYmUgbmljZSB0byBoYXZlIGEgY2xlYXIgaWRlYSBvZiB3aGF0
J3MgaGFwcGVuZWQgYmVmb3JlIGl0IGFsbCB3ZW50IGJlbGx5LXVwLiBJJ2xsIGdldCBiYWNrIHRv
IHlvdS4NCg0KTUguDQo=


From win-pv-devel-bounces@lists.xenproject.org Tue Mar 22 11:13:31 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Tue, 22 Mar 2022 11:13:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.293365.498360 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nWcS9-0000FH-P5; Tue, 22 Mar 2022 11:13:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 293365.498360; Tue, 22 Mar 2022 11:13:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nWcS9-0000FA-M9; Tue, 22 Mar 2022 11:13:29 +0000
Received: by outflank-mailman (input) for mailman id 293365;
 Tue, 22 Mar 2022 11:13:27 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mH5U=UB=citrix.com=prvs=073f57e0a=martin.harvey@srs-se1.protection.inumbo.net>)
 id 1nWcS7-0000F4-Ek
 for win-pv-devel@lists.xenproject.org; Tue, 22 Mar 2022 11:13:27 +0000
Received: from esa5.hc3370-68.iphmx.com (esa5.hc3370-68.iphmx.com
 [216.71.155.168]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 165b9230-a9d1-11ec-a405-831a346695d4;
 Tue, 22 Mar 2022 12:13:24 +0100 (CET)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 165b9230-a9d1-11ec-a405-831a346695d4
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1647947604;
  h=from:to:cc:subject:date:message-id:mime-version:
   content-transfer-encoding;
  bh=cTPM4tPHQyGGGjWwg6JtBMLYEAx9z7TWJ2vLk7m+HdA=;
  b=ZhG4QUQ8+dWH/vGDZNDoLxeEkc36rdtemzyqLskBvfkxZzAG5BJkphq3
   WVlpkk63t9YDqJS+K9fXWREQY8q/941jbxSBe3Oho8lvi5oVaBCURLvCA
   V+ppsfKXDZyMW4oVQS68Pk0qjFe8jnH7MVH9BUqnGUP6kksstRZRc/6gM
   I=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
X-SBRS: 5.1
X-MesageID: 66304701
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:4VWoaa3vCjx2rkHTpfbD5bVxkn2cJEfYwER7XKvMYLTBsI5bpz0Cz
 DcXCz2CPK2JYGv0fowgaY6+9RgG6MPTzN9nGVRrpC1hF35El5HIVI+TRqvS04J+DSFhoGZPt
 Zh2hgzodZhsJpPkjk7xdOCn9xGQ7InQLlbGILes1htZGEk1EE/NtTo5w7Rj2tUy24Dja++wk
 YiaT/P3aQfNNwFcagr424rbwP+4lK2v0N+wlgVWicFj5DcypVFMZH4sDfjZw0/DaptVBoaHq
 9Prl9lVyI97EyAFUbtJmp6jGqEDryW70QKm0hK6UID66vROS7BbPg/W+5PwZG8O4whlkeydx
 /1Q5ZaJaz0jAZHtkf4PeQZeUAFYHL1/reqvzXiX6aR/zmXDenrohf5vEFs3LcsT/eMf7WNmr
 KJCbmpXN1ba2rzwkOnTpupE36zPKOHuNZkDu3cmzTjDE/s3aZvCX7/L9ZlT2zJYasVmQ6iFP
 5ZGMWMHgBLoZy9/K1sGLcIEgdysozrlNBtZ8Gq8uv9ii4TU5FMoi+W8WDbPQfSVRMMQhljdq
 m/Y8mDRBhABKMfZ2TeD6mirhOLEgWX8Qo16KVGj3qc02hvJnDVVUUBIEwvgyRWktqKgc88PI
 XU2wCARkYk77l6BUd++fzuZsEfR63bwROFsO+E97QiMzI/d7ACYGnUIQ1Z9VTA2iCMlbWd0j
 wHUxrsFERQq6eTIEizFqt94uBvoYUAowXk+iTjopOfvy/3qu8kNgx3GVb6P+4bl34SuSVkcL
 91nxRXSZon/b+ZWjs1XHnid2lpAQ6QlqCZvvW07uUr/smtEiHaNPdDA1LQixa8owHylZleAp
 mMYvMOV8foDC5qA/ATUHrlSTOH2uqnUamOB6bKKI3XH327xk5JEVdoNiAyS2W8zappUEdMXS
 BG7VfxtCG97YyLxMP4fj3OZAMU216nwfekJpdiPBueilqNZLVfdlAk3PBb49zm0zCAEzPFuU
 b/GIJ3EJStLVsxaIM+eGr51PUkDnXtlmws+hPnTknya7FZpTCXMGOlfbwfWMLxRAWHtiFy9z
 uuz/vCik313ONASqAGNmWLPBTjm9UQGOK0=
IronPort-HdrOrdr: A9a23:/xNfQa5cP9GQLaWIKAPXwMbXdLJyesId70hD6qhwISY1TiX+rb
 HIoB17726MtN9/YgBCpTntAsa9qBDnhPpICOsqTNWftWDd0QPCRuwP0WKL+UyHJ8SUzI5gPM
 lbHZSWcOeAaGRHsQ==
X-IronPort-AV: E=Sophos;i="5.90,201,1643691600"; 
   d="scan'208";a="66304701"
From: Martin Harvey <Martin.Harvey@citrix.com>
To: <win-pv-devel@lists.xenproject.org>
CC: Martin Harvey <Martin.Harvey@citrix.com>, Martin Harvey
	<martin.harvey@citrix.com>
Subject: [PATCH] Windows PV drivers fail to set up RSS when vCPUs > 8
Date: Tue, 22 Mar 2022 11:12:44 +0000
Message-ID: <20220322111244.764-1-Martin.Harvey@citrix.com>
X-Mailer: git-send-email 2.25.0.windows.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

The driver only supports at most 8 queues, however Windows
can decide to assign vCPU numbers starting from a non-zero
offset. E.g. vCPU 8,9,10,11 could get assigned to a device
if you have more than one NIC. The total number of vCPUs
used by a single device is still less than 8, but the vCPU
indexes themselves can be greater than 8. The code
previously incorrectly assumed that individual vCPU
indexes cannot exceed 8, however a 1:1 mapping between
vCPU indexes and queues seems to only exist when using
a single NIC.

Signed-off-by: Martin Harvey <martin.harvey@citrix.com>
---
 src/xenvif/receiver.c | 168 +++++++++++++++++++++++++++++++++++++-----
 1 file changed, 148 insertions(+), 20 deletions(-)

diff --git a/src/xenvif/receiver.c b/src/xenvif/receiver.c
index 10ac6f5..0c7b32a 100644
--- a/src/xenvif/receiver.c
+++ b/src/xenvif/receiver.c
@@ -106,6 +106,7 @@ typedef struct _XENVIF_RECEIVER_RING {
     PLIST_ENTRY                 PacketQueue;
     KDPC                        QueueDpc;
     ULONG                       QueueDpcs;
+    PROCESSOR_NUMBER            TargetProcessor;
     LIST_ENTRY                  PacketComplete;
     XENVIF_RECEIVER_HASH        Hash;
 } XENVIF_RECEIVER_RING, *PXENVIF_RECEIVER_RING;
@@ -2498,6 +2499,9 @@ __ReceiverRingInitialize(
 
     KeInitializeThreadedDpc(&(*Ring)->QueueDpc, ReceiverRingQueueDpc, *Ring);
 
+    status = KeGetProcessorNumberFromIndex((*Ring)->Index, &(*Ring)->TargetProcessor);
+    ASSERT(NT_SUCCESS(status));
+
     return STATUS_SUCCESS;
 
 fail7:
@@ -2550,6 +2554,45 @@ fail1:
     return status;
 }
 
+static NTSTATUS
+__ReceiverRingSetAffinity(
+    IN  PXENVIF_RECEIVER_RING   Ring,
+    IN  PPROCESSOR_NUMBER       Processor
+    )
+{
+    PXENVIF_RECEIVER            Receiver;
+    PXENVIF_FRONTEND            Frontend;
+    NTSTATUS status;
+
+    status = STATUS_INVALID_PARAMETER;
+    if ((Ring == NULL) || (Processor == NULL))
+        goto fail1;
+
+    Receiver = Ring->Receiver;
+    Frontend = Receiver->Frontend;
+
+    /* Always update ring target processor
+       Actually set affinities if frontend override not present.
+       Re-bind event-channel if already connected */
+
+    __ReceiverRingAcquireLock(Ring);
+
+    Ring->TargetProcessor = *Processor;
+
+    /* Don't rebind event channel at this point. */
+    KeSetTargetProcessorDpcEx(&Ring->PollDpc, &Ring->TargetProcessor);
+    KeSetTargetProcessorDpcEx(&Ring->QueueDpc, &Ring->TargetProcessor);
+
+    __ReceiverRingReleaseLock(Ring);
+
+    return STATUS_SUCCESS;
+
+fail1:
+    Error("fail1 (%08x)\n", status);
+
+    return status;
+}
+
 static FORCEINLINE NTSTATUS
 __ReceiverRingConnect(
     IN  PXENVIF_RECEIVER_RING   Ring
@@ -2560,7 +2603,6 @@ __ReceiverRingConnect(
     PFN_NUMBER                  Pfn;
     CHAR                        Name[MAXNAMELEN];
     ULONG                       Index;
-    PROCESSOR_NUMBER            ProcNumber;
     NTSTATUS                    status;
 
     Receiver = Ring->Receiver;
@@ -2637,16 +2679,17 @@ __ReceiverRingConnect(
     if (Ring->Channel == NULL)
         goto fail6;
 
-    status = KeGetProcessorNumberFromIndex(Ring->Index, &ProcNumber);
-    ASSERT(NT_SUCCESS(status));
+    status = XENBUS_EVTCHN(Bind,
+                            &Receiver->EvtchnInterface,
+                            Ring->Channel,
+                            Ring->TargetProcessor.Group,
+                            Ring->TargetProcessor.Number);
+    if (!NT_SUCCESS(status))
+        Warning("Cound not set initial receiver ring affinity: 0x%x\n", status);
+    /* You haven't specifically asked for an affinity yet, so just warn. */
 
-    KeSetTargetProcessorDpcEx(&Ring->PollDpc, &ProcNumber);
-
-    (VOID) XENBUS_EVTCHN(Bind,
-                         &Receiver->EvtchnInterface,
-                         Ring->Channel,
-                         ProcNumber.Group,
-                         ProcNumber.Number);
+    KeSetTargetProcessorDpcEx(&Ring->PollDpc, &Ring->TargetProcessor);
+    KeSetTargetProcessorDpcEx(&Ring->QueueDpc, &Ring->TargetProcessor);
 
     (VOID) XENBUS_EVTCHN(Unmask,
                          &Receiver->EvtchnInterface,
@@ -2665,11 +2708,6 @@ __ReceiverRingConnect(
     if (!NT_SUCCESS(status))
         goto fail7;
 
-    status = KeGetProcessorNumberFromIndex(Ring->Index, &ProcNumber);
-    ASSERT(NT_SUCCESS(status));
-
-    KeSetTargetProcessorDpcEx(&Ring->QueueDpc, &ProcNumber);
-
     return STATUS_SUCCESS;
 
 fail7:
@@ -3917,6 +3955,56 @@ fail1:
     return status;
 }
 
+static NTSTATUS
+__ReceiverSetQueueAffinities(
+    IN  PXENVIF_RECEIVER        Receiver,
+    IN  PPROCESSOR_NUMBER       QueueAffinities,
+    IN  ULONG                   Count
+    )
+{
+    PXENVIF_FRONTEND        Frontend;
+    ULONG                   Index;
+    NTSTATUS                status;
+    KIRQL                   Irql;
+
+    Frontend = Receiver->Frontend;
+
+    status = STATUS_INVALID_PARAMETER;
+
+    if (QueueAffinities == NULL)
+        goto fail1;
+
+    if (Count > FrontendGetNumQueues(Frontend))
+        goto fail2;
+
+    KeRaiseIrql(DISPATCH_LEVEL, &Irql);
+
+    for (Index = 0; Index < Count; Index++) {
+        PXENVIF_RECEIVER_RING   Ring = Receiver->Ring[Index];
+
+        status = __ReceiverRingSetAffinity(Ring, &QueueAffinities[Index]);
+        if (!NT_SUCCESS(status))
+            goto fail3;
+    }
+
+    KeLowerIrql(Irql);
+
+    return STATUS_SUCCESS;
+
+fail3:
+    KeLowerIrql(Irql);
+
+    Error("fail3\n");
+
+fail2:
+    Error("fail2\n");
+
+fail1:
+    Error("fail1 (%08x)\n", status);
+
+    return status;
+}
+
 NTSTATUS
 ReceiverUpdateHashMapping(
     IN  PXENVIF_RECEIVER    Receiver,
@@ -3926,10 +4014,15 @@ ReceiverUpdateHashMapping(
 {
     PXENVIF_FRONTEND        Frontend;
     PULONG                  QueueMapping;
+    PPROCESSOR_NUMBER       QueueAffinities;
     ULONG                   NumQueues;
+    ULONG                   QueuesDetermined;
+    ULONG                   QIndex;
     ULONG                   Index;
+    BOOLEAN                 MapEntryDone;
     NTSTATUS                status;
 
+
     Frontend = Receiver->Frontend;
 
     QueueMapping = __ReceiverAllocate(sizeof (ULONG) * Size);
@@ -3939,26 +4032,61 @@ ReceiverUpdateHashMapping(
         goto fail1;
 
     NumQueues = FrontendGetNumQueues(Frontend);
+    QueuesDetermined = 0;
+
+    QueueAffinities = __ReceiverAllocate(sizeof(PROCESSOR_NUMBER) * NumQueues);
+    if (QueueAffinities == NULL)
+        goto fail2;
 
     status = STATUS_INVALID_PARAMETER;
+    /* N^Squared-ish, but performed infrequently */
     for (Index = 0; Index < Size; Index++) {
-        QueueMapping[Index] = KeGetProcessorIndexFromNumber(&ProcessorMapping[Index]);
-
-        if (QueueMapping[Index] >= NumQueues)
-            goto fail2;
+        MapEntryDone = FALSE;
+        /* Existing queue meets affinity requirement for the mapping at this index? */
+        for (QIndex = 0; QIndex < QueuesDetermined; QIndex++) {
+            if ((QueueAffinities[QIndex].Group == ProcessorMapping[Index].Group) &&
+                (QueueAffinities[QIndex].Number == ProcessorMapping[Index].Number)) {
+                QueueMapping[Index] = QIndex;
+                MapEntryDone = TRUE;
+            }
+        }
+        if (!MapEntryDone) {
+            /* New queue "allocation", with new affinity, if possible */
+            if (QueuesDetermined < NumQueues) {
+                QIndex = QueuesDetermined;
+                QueueAffinities[QIndex] = ProcessorMapping[Index];
+                QueueMapping[Index] = QIndex;
+                QueuesDetermined ++;
+            } else {
+                goto fail3;
+            }
+        }
     }
 
     status = FrontendSetHashMapping(Frontend, QueueMapping, Size);
     if (!NT_SUCCESS(status))
-        goto fail3;
+        goto fail4;
+
+    status = __ReceiverSetQueueAffinities(Receiver, QueueAffinities, QueuesDetermined);
+    if (!NT_SUCCESS(status))
+        goto fail5;
 
     __ReceiverFree(QueueMapping);
+    __ReceiverFree(QueueAffinities);
 
     return STATUS_SUCCESS;
 
+fail5:
+    Error("fail5\n");
+
+fail4:
+    Error("fail4\n");
+
 fail3:
     Error("fail3\n");
 
+    __ReceiverFree(QueueAffinities);
+
 fail2:
     Error("fail2\n");
 
-- 
2.25.0.windows.1



From win-pv-devel-bounces@lists.xenproject.org Tue Mar 22 12:06:10 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Tue, 22 Mar 2022 12:06:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.293413.498469 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nWdH7-0002lN-Vz; Tue, 22 Mar 2022 12:06:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 293413.498469; Tue, 22 Mar 2022 12:06:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nWdH7-0002lG-Sy; Tue, 22 Mar 2022 12:06:09 +0000
Received: by outflank-mailman (input) for mailman id 293413;
 Tue, 22 Mar 2022 12:06:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UN57=UB=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1nWdH7-0002lA-9Z
 for win-pv-devel@lists.xenproject.org; Tue, 22 Mar 2022 12:06:09 +0000
Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com
 [2a00:1450:4864:20::32a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 748d5270-a9d8-11ec-a405-831a346695d4;
 Tue, 22 Mar 2022 13:06:07 +0100 (CET)
Received: by mail-wm1-x32a.google.com with SMTP id i186so1262517wma.3
 for <win-pv-devel@lists.xenproject.org>; Tue, 22 Mar 2022 05:06:07 -0700 (PDT)
Received: from [10.139.200.51] ([54.239.6.189])
 by smtp.gmail.com with ESMTPSA id
 bg18-20020a05600c3c9200b0037c2ef07493sm2020092wmb.3.2022.03.22.05.06.05
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 22 Mar 2022 05:06:06 -0700 (PDT)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 748d5270-a9d8-11ec-a405-831a346695d4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=message-id:date:mime-version:user-agent:reply-to:subject
         :content-language:to:cc:references:from:in-reply-to
         :content-transfer-encoding;
        bh=hr3EY6rF45ArM3nCFdDnDM47aalu5JiXrbNnpwMSMys=;
        b=dDi9aqw6X/BwLGh+jURkD/JPO34SvK4e9Wrs2q5tMz8cJAferfBEukAGoFX2g826TD
         AP1O2bz3/k+biT6Jiky3jpK5wen+NFJeByiDr8Rq/BK5Jj2OVlz+mEUdH0nPB956u+vr
         nb9vh3blxr7wx/MFXKfRMJqwKSeaVTDXZ1EQ+WxjUxGQ4MVi3JfT50RUHIAwXJ2rSsUe
         dqVr7Gx9jVhssojaEwF8+ya0LJo2ApQKXwZZgt5sQmXNfRMGr0OYBdLTbnixvvOu1Wl/
         ZccsTTKWL27PMo40CApYRy44UOgzjLJF3ufXwlU71xsLrMkHxcq2+8g5BzilUIaLt/0Q
         2SjA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:message-id:date:mime-version:user-agent:reply-to
         :subject:content-language:to:cc:references:from:in-reply-to
         :content-transfer-encoding;
        bh=hr3EY6rF45ArM3nCFdDnDM47aalu5JiXrbNnpwMSMys=;
        b=y76qYos1M0jNO09GQ17aH3ujwi3T1F/W05Ud80di4/VAALrVuL9oJyB7zmIt56pBYK
         JNf1xM0MrcQA7+DwuHbVjQ2x8QZmP835YZNDU4KwCLJXsKzBBB4zVK3iIQwdF2f9e++F
         bqwSMvl5L2J3+IeSkobPwfXuIQ8gUHFM4O4rdcgEORR6+10nH9PjduXNUoyAHSgYLz6W
         RKnhGpuYmuWG0I8OS470cJq7Cn+NDyp9cqABIifXMhzJfeuxp8PdWiTh95tZ+dS4gCmv
         bP8gJVbRSfTfdOcnkhzHo8g8Mby+py65jd/Obc1v7/IzpvCXxGzipfahfJGadagZNxmz
         0Vcw==
X-Gm-Message-State: AOAM532V86sQwFNIQjWYTekRLW/XBAe+CkSZXvBuGYqQD9MVE8C5Tfax
	22D8A/DnSASzBxUUuPGXIbA=
X-Google-Smtp-Source: ABdhPJzBYD1n1hj9UQKzovC7MCVA5fjpWh3h8+88ynFdQnBZox+kZjo8OV8qsR2Gf/xxtR0Xc1Xk0g==
X-Received: by 2002:a7b:cd03:0:b0:37b:fc83:a4e2 with SMTP id f3-20020a7bcd03000000b0037bfc83a4e2mr3455474wmj.193.1647950767021;
        Tue, 22 Mar 2022 05:06:07 -0700 (PDT)
Message-ID: <4393f6d9-5d74-65ee-4bc1-ebb0d49433e9@gmail.com>
Date: Tue, 22 Mar 2022 13:06:05 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.7.0
Reply-To: paul@xen.org
Subject: Re: [PATCH] Windows PV drivers fail to set up RSS when vCPUs > 8
Content-Language: en-US
To: Edwin Torok <edvin.torok@citrix.com>,
 Martin Harvey <martin.harvey@citrix.com>
Cc: "paul@xen.org" <paul@xen.org>,
 "win-pv-devel@lists.xenproject.org" <win-pv-devel@lists.xenproject.org>
References: <20220314152746.1712-1-Martin.Harvey@citrix.com>
 <e81b19af-067b-1a98-7952-f06c101a2418@gmail.com>
 <MW4PR03MB63476907571ADA5598BE441E9A169@MW4PR03MB6347.namprd03.prod.outlook.com>
 <7FDE677E-E1C9-4FC1-931F-1FE102558E17@citrix.com>
From: "Durrant, Paul" <xadimgnik@gmail.com>
In-Reply-To: <7FDE677E-E1C9-4FC1-931F-1FE102558E17@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 21/03/2022 17:42, Edwin Torok wrote:
> 
> 
>> On 21 Mar 2022, at 16:06, Martin Harvey <martin.harvey@citrix.com> wrote:
>>
>>
>>
>> -----Original Message-----
>> From: win-pv-devel <win-pv-devel-bounces@lists.xenproject.org> On Behalf Of Durrant, Paul
>> Sent: 19 March 2022 17:39
>> To: win-pv-devel@lists.xenproject.org
>> Subject: Re: [PATCH] Windows PV drivers fail to set up RSS when vCPUs > 8
>>
>> [CAUTION - EXTERNAL EMAIL] DO NOT reply, click links, or open attachments unless you have verified the sender and know the content is safe.
>>
>>
>>> I think we need a bit more in the commit comment. What is the nature of the failure... and does XENNET advertise more than 8 queues, so will the situation ever arise? Linux certainly tops out at 8 queues.
>>
>> I don't believe that XenNet ever advertises more than 8 queues, but that's not quite the same as supporting more than 8 vCPU's
>>
>> Perhaps something in the comment like: "Mapping between Queues and VPU's fails for more than 8 VCPU's because the base of the indirection is always considered to be zero, and the mapping is always performed on a direct vCPU number basis"
>>
>> Would that adequately summarise the problem? (It's not easy to explain succinctly in English!)
> 
> Here is another attempt at explaining it, perhaps you could put your one sentence in the commit title, and the one below in the commit body?
> 
> The driver only supports at most 8 queues, however Windows can decide to assign vCPU numbers starting from a non-zero offset.
> E.g. vCPU 8,9,10,11 could get assigned to a device if you have more than one NIC. The total number of vCPUs used by a single device is still less than 8, but the vCPU indexes themselves can be greater than 8.
> The code previously incorrectly assumed that individual vCPU indexes cannot exceed 8, however a 1:1 mapping between vCPU indexes and queues seems to only exist when using a single NIC.
> [Full example below]

Ok, I understand now. So what we want is a more flexible queue -> CPU 
mapping. I'll apply the patch locally to my dev branch and take a closer 
look.

Thanks,

   Paul

> 
>>
>> MH.
>>
>> This summary helpfully provided by Edvin Torok.
>>
>>
>> On a VM with >8 vCPUs RSS might not work because the driver fails to set up the indirection table.
>>
>> This causes the VM to only be able to reach 12.4Gbit/s with 'iperf3 -P 8', instead of 16-18Gbit/s with a working RSS setup:
>>
>> This can be easily reproduced if you give a VM 32 vCPUs and create 3 network interfaces. Windows will assign 0-3 to one network interface, 4-7 to next, and will try 8-12 I think for next but the driver rejects that:
>>
>> PS C:\Program Files\CItrix\XenTools\Diagnostics> Get-NetAdapterRSS
>>
>>
>> Name                                            : Ethernet 5
>> InterfaceDescription                            : XenServer PV Network Device #2
>> Enabled                                         : True
>> NumberOfReceiveQueues                           : 8
>> Profile                                         : NUMAStatic
>> BaseProcessor: [Group:Number]                   : 0:0
>> MaxProcessor: [Group:Number]                    : 0:31
>> MaxProcessors                                   : 4
>> RssProcessorArray: [Group:Number/NUMA Distance] : 0:0/0  0:1/0  0:2/0  0:3/0  0:4/0  0:5/0  0:6/0  0:7/0
>>                                                   0:8/0  0:9/0  0:10/0  0:11/0  0:12/0  0:13/0  0:14/0  0:15/0
>>                                                   0:16/0  0:17/0  0:18/0  0:19/0  0:20/0  0:21/0  0:22/0  0:23/0
>>                                                   0:24/0  0:25/0  0:26/0  0:27/0  0:28/0  0:29/0  0:30/0  0:31/0
>> IndirectionTable: [Group:Number]                :
>>
>> Name                                            : Ethernet 4
>> InterfaceDescription                            : XenServer PV Network Device #1
>> Enabled                                         : True
>> NumberOfReceiveQueues                           : 8
>> Profile                                         : NUMAStatic
>> BaseProcessor: [Group:Number]                   : 0:0
>> MaxProcessor: [Group:Number]                    : 0:31
>> MaxProcessors                                   : 4
>> RssProcessorArray: [Group:Number/NUMA Distance] : 0:0/0  0:1/0  0:2/0  0:3/0  0:4/0  0:5/0  0:6/0  0:7/0
>>                                                   0:8/0  0:9/0  0:10/0  0:11/0  0:12/0  0:13/0  0:14/0  0:15/0
>>                                                   0:16/0  0:17/0  0:18/0  0:19/0  0:20/0  0:21/0  0:22/0  0:23/0
>>                                                   0:24/0  0:25/0  0:26/0  0:27/0  0:28/0  0:29/0  0:30/0  0:31/0
>> IndirectionTable: [Group:Number]                : 0:4   0:5     0:6     0:7     0:4     0:5     0:6     0:7
>>                                                   0:4   0:5     0:6     0:7     0:4     0:5     0:6     0:7
>>                                                   0:4   0:5     0:6     0:7     0:4     0:5     0:6     0:7
>>                                                   0:4   0:5     0:6     0:7     0:4     0:5     0:6     0:7
>>                                                   0:4   0:5     0:6     0:7     0:4     0:5     0:6     0:7
>>                                                   0:4   0:5     0:6     0:7     0:4     0:5     0:6     0:7
>>                                                   0:4   0:5     0:6     0:7     0:4     0:5     0:6     0:7
>>                                                   0:4   0:5     0:6     0:7     0:4     0:5     0:6     0:7
>>                                                   0:4   0:5     0:6     0:7     0:4     0:5     0:6     0:7
>>                                                   0:4   0:5     0:6     0:7     0:4     0:5     0:6     0:7
>>                                                   0:4   0:5     0:6     0:7     0:4     0:5     0:6     0:7
>>                                                   0:4   0:5     0:6     0:7     0:4     0:5     0:6     0:7
>>                                                   0:4   0:5     0:6     0:7     0:4     0:5     0:6     0:7
>>                                                   0:4   0:5     0:6     0:7     0:4     0:5     0:6     0:7
>>                                                   0:4   0:5     0:6     0:7     0:4     0:5     0:6     0:7
>>                                                   0:4   0:5     0:6     0:7     0:4     0:5     0:6     0:7
>>
>> Name                                            : Ethernet 3
>> InterfaceDescription                            : XenServer PV Network Device #0
>> Enabled                                         : True
>> NumberOfReceiveQueues                           : 8
>> Profile                                         : NUMAStatic
>> BaseProcessor: [Group:Number]                   : 0:0
>> MaxProcessor: [Group:Number]                    : 0:31
>> MaxProcessors                                   : 4
>> RssProcessorArray: [Group:Number/NUMA Distance] : 0:0/0  0:1/0  0:2/0  0:3/0  0:4/0  0:5/0  0:6/0  0:7/0
>>                                                   0:8/0  0:9/0  0:10/0  0:11/0  0:12/0  0:13/0  0:14/0  0:15/0
>>                                                   0:16/0  0:17/0  0:18/0  0:19/0  0:20/0  0:21/0  0:22/0  0:23/0
>>                                                   0:24/0  0:25/0  0:26/0  0:27/0  0:28/0  0:29/0  0:30/0  0:31/0
>> IndirectionTable: [Group:Number]                : 0:0   0:1     0:2     0:3     0:0     0:1     0:2     0:3
>>                                                   0:0   0:1     0:2     0:3     0:0     0:1     0:2     0:3
>>                                                   0:0   0:1     0:2     0:3     0:0     0:1     0:2     0:3
>>                                                   0:0   0:1     0:2     0:3     0:0     0:1     0:2     0:3
>>                                                   0:0   0:1     0:2     0:3     0:0     0:1     0:2     0:3
>>                                                   0:0   0:1     0:2     0:3     0:0     0:1     0:2     0:3
>>                                                   0:0   0:1     0:2     0:3     0:0     0:1     0:2     0:3
>>                                                   0:0   0:1     0:2     0:3     0:0     0:1     0:2     0:3
>>                                                   0:0   0:1     0:2     0:3     0:0     0:1     0:2     0:3
>>                                                   0:0   0:1     0:2     0:3     0:0     0:1     0:2     0:3
>>                                                   0:0   0:1     0:2     0:3     0:0     0:1     0:2     0:3
>>                                                   0:0   0:1     0:2     0:3     0:0     0:1     0:2     0:3
>>                                                   0:0   0:1     0:2     0:3     0:0     0:1     0:2     0:3
>>                                                   0:0   0:1     0:2     0:3     0:0     0:1     0:2     0:3
>>                                                   0:0   0:1     0:2     0:3     0:0     0:1     0:2     0:3
>>                                                   0:0   0:1     0:2     0:3     0:0     0:1     0:2     0:3
>> There is a builtin hardcoded limit of 8 queues, which is fine, but that should be completely unrelated to CPU numbers! (the total number of CPUs assigned to a NIC should be <=8, sure).
>>
>> Potential code causing issue in xenvif receiver.c:
>>
>> for (Index = 0; Index < Size; Index++) {
>>           QueueMapping[Index] = KeGetProcessorIndexFromNumber(&ProcessorMapping[Index]);
>>           if (QueueMapping[Index] >= NumQueues)
>>               goto fail2;
>>       }
>>
>> (there is also a problem that the code assumes that group number is always 0. For now that is true, but might change if we implement vNUMA in the future).
>>
>> Jun  2 09:54:57 prost qemu-dm-41[30818]: 30818@1622627697.450233:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
>> Jun  2 09:54:57 prost qemu-dm-41[30818]: 30818@1622627697.450320:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
>> Jun  2 09:54:57 prost qemu-dm-41[30818]: 30818@1622627697.452097:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
>> Jun  2 09:54:57 prost qemu-dm-41[30818]: 30818@1622627697.452180:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
>> Jun  2 09:56:14 prost qemu-dm-41[30818]: 30818@1622627774.374713:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
>> Jun  2 09:56:14 prost qemu-dm-41[30818]: 30818@1622627774.374798:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
>> Jun  2 09:56:14 prost qemu-dm-41[30818]: 30818@1622627774.377121:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
>> Jun  2 09:56:14 prost qemu-dm-41[30818]: 30818@1622627774.377203:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
>> Jun  2 09:59:00 prost qemu-dm-42[1106]: 1106@1622627940.672941:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
>> Jun  2 09:59:00 prost qemu-dm-42[1106]: 1106@1622627940.673058:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
>> Jun  2 09:59:00 prost qemu-dm-42[1106]: 1106@1622627940.675891:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
>> Jun  2 09:59:00 prost qemu-dm-42[1106]: 1106@1622627940.675993:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
>> Jun  2 09:59:39 prost qemu-dm-43[4074]: 4074@1622627979.363892:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
>> Jun  2 09:59:39 prost qemu-dm-43[4074]: 4074@1622627979.364008:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
>> Jun  2 09:59:39 prost qemu-dm-43[4074]: 4074@1622627979.365861:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
>> Jun  2 09:59:39 prost qemu-dm-43[4074]: 4074@1622627979.365949:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
>> Jun  2 10:04:41 prost qemu-dm-45[9705]: 9705@1622628281.935871:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
>> Jun  2 10:04:41 prost qemu-dm-45[9705]: 9705@1622628281.935965:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
>> Jun  2 10:04:41 prost qemu-dm-45[9705]: 9705@1622628281.937849:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
>> Jun  2 10:04:41 prost qemu-dm-45[9705]: 9705@1622628281.937918:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
>> Jun  2 10:05:00 prost qemu-dm-46[11484]: 11484@1622628300.973487:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
>> Jun  2 10:05:00 prost qemu-dm-46[11484]: 11484@1622628300.973588:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
>> Jun  2 10:05:00 prost qemu-dm-46[11484]: 11484@1622628300.976554:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
>> Jun  2 10:05:00 prost qemu-dm-46[11484]: 11484@1622628300.976650:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
>> Jun  2 10:22:54 prost qemu-dm-49[21901]: 21901@1622629374.720769:xen_platform_log xen platform: xenvif|PdoGetInterfaceGuid: fail1 (c0000034)
>> Jun  2 10:22:55 prost qemu-dm-49[21901]: 21901@1622629375.194122:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
>> Jun  2 10:22:55 prost qemu-dm-49[21901]: 21901@1622629375.194231:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
>> Jun  2 10:22:55 prost qemu-dm-49[21901]: 21901@1622629375.196726:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
>> Jun  2 10:22:55 prost qemu-dm-49[21901]: 21901@1622629375.196825:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
>> Jun  2 10:23:38 prost qemu-dm-50[24509]: 24509@1622629418.530046:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
>> Jun  2 10:23:38 prost qemu-dm-50[24509]: 24509@1622629418.530115:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
>> Jun  2 10:23:38 prost qemu-dm-50[24509]: 24509@1622629418.531811:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
>> Jun  2 10:23:38 prost qemu-dm-50[24509]: 24509@1622629418.531888:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
>> Jun  2 10:30:28 prost qemu-dm-51[28530]: 28530@1622629828.510968:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
>> Jun  2 10:30:28 prost qemu-dm-51[28530]: 28530@1622629828.511050:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
>> Jun  2 10:30:28 prost qemu-dm-51[28530]: 28530@1622629828.513570:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
>> Jun  2 10:30:28 prost qemu-dm-51[28530]: 28530@1622629828.513691:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
>> Jun  2 10:45:43 prost qemu-dm-52[2889]: 2889@1622630743.573791:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
>> Jun  2 10:45:43 prost qemu-dm-52[2889]: 2889@1622630743.573904:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
>> Jun  2 10:45:43 prost qemu-dm-52[2889]: 2889@1622630743.576188:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
>> Jun  2 10:45:43 prost qemu-dm-52[2889]: 2889@1622630743.576298:xen_platform_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
>>
>> I tested with both Win10 and Windows Server 2016, with various CPU topologies (e.g. 12 vCPUs, all on one socket shows same issue once windows starts assigning CPUs>8)
>>
>> A workaround is to set MaxProcessorNumber to 7, though obviously this will limit scalability since vCPUs > 8 won't be used even if you have multiple VIFs:
>>
>>
>>
> 



From win-pv-devel-bounces@lists.xenproject.org Tue Mar 22 12:46:58 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Tue, 22 Mar 2022 12:46:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.293101.498540 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nWdub-00019i-Gq; Tue, 22 Mar 2022 12:46:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 293101.498540; Tue, 22 Mar 2022 12:46:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nWdub-00019b-DM; Tue, 22 Mar 2022 12:46:57 +0000
Received: by outflank-mailman (input) for mailman id 293101;
 Mon, 21 Mar 2022 16:42:24 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XmoN=UA=citrix.com=prvs=07239607a=edvin.torok@srs-se1.protection.inumbo.net>)
 id 1nWL6u-0008Uv-9G
 for win-pv-devel@lists.xenproject.org; Mon, 21 Mar 2022 16:42:24 +0000
Received: from esa3.hc3370-68.iphmx.com (esa3.hc3370-68.iphmx.com
 [216.71.145.155]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id dfce4f49-a935-11ec-a405-831a346695d4;
 Mon, 21 Mar 2022 17:42:21 +0100 (CET)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dfce4f49-a935-11ec-a405-831a346695d4
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1647880941;
  h=from:to:cc:subject:date:message-id:references:
   in-reply-to:content-id:content-transfer-encoding:
   mime-version;
  bh=iOE+DgFu8PMWtK+e/MtNpPf+pcKCozB8dgfrlyyJpoI=;
  b=H3ax2mS4p6ui3AsnOEhcFKLKGQ+w5E3P55JFE1EgqGbCL4yMIKId+Stm
   wFQw9LVtHJG7clxByA9DO241FsRpqBT/sZ7tJEaPjp0FA9DPy3di2hA4D
   G5vdWmq9vYh8Nl37XCb7qp6K0ojLOldBDv3QDL4P6yzLX0Z2era+Xbg8V
   8=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
X-SBRS: 5.1
X-MesageID: 66782867
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:pmC/UKyeI/hcNCKXvnF6t+dIxirEfRIJ4+MujC+fZmUNrF6WrkUEx
 2EWCGuPO/uIMGqmctwjYNi/8UMOvceAztU3QAU+qSAxQypGp/SeCIXCJC8cHc8zwu4v7q5Dx
 59DAjUVBJlsFhcwnj/0bv656yMUOZigHtIQMsadUsxKbVIiGX9JZS5LwbZj2NYz24HhWWthh
 PupyyHhEA79s9JLGjp8B5Kr8HuDa9yr5Vv0FnRnDRx6lAe2e0s9VfrzFonoR5fMeaFGH/bSe
 gr25OrRElU1XfsaIojNfr7TKiXmS1NJVOSEoiI+t6OK2nCuqsGuu0qS2TV1hUp/0l20c95NJ
 Npls8KiZFY0MYP2ie0mQjNzTQdDZLFn0eqSSZS/mZT7I0zudnLtx7NlDV0sPJ1e8eFyaY1M3
 aVGcnZXNEnF3r/ohuLgIgVvrp1LwM3DMY8Bu3cm0TbDCvUOSpHfWaTao9Rf2V/cg+gQQ6mOP
 ZpHNlKDajzbTztVG3IVF6szgcT5vGKkdhN0i32s8P9fD2/7k1UqjemF3MDuUsOLQ4BNgAOUq
 3zL+0z9AwoGL5qPxDyd6HWui+TT2yThV+o6DLSl8tZ6jVvVwXYcYCD6TnPi/6P/0BTnHYsCd
 QpEoULCsJTe6mStEMvQDgC3jkSVtyUNcfEONdBj9CCkn/+8DxmiOkAISTtIadoDvcAwRCA32
 lLho+4FFQCDo5XOFyvDq+78QSeafHFMcDRcPXNsoR4tuYGLnW0lsv7Yoj+P+oaRh8a9Jzz/y
 iviQMMW1+RK1p5jO0lWEDn6b9OQSnrhE1VdCub/BDvNAuZFiGmNPdDA1LQjxawcRLt1t3HY1
 JT+p+CQ7foVEbaGnzGXTeMGEdmBvqjZb22F2Qc2T8B6plxBHkJPmqgKvVlDyLpBaJ5YKVcFn
 meN0e+u2HOjFCTzNvImC25AI88r0bLhBbzYugP8NbJzjmxKXFbfpklGPBfIt0i0yRREufxva
 P+zLJf3ZV5HWPsP8dZDb7pEuVPd7ntlnj27qFGS50nP7Idyk1bOEOZbagTSNr5lhE5GyS2Mm
 +ti2wKx40w3eMX1YzXN8J5VKlYPLHMhAovxpdARfemGSjeK0kl6YxMN6dvNo7BYopk=
IronPort-HdrOrdr: A9a23:HRtVBa4TX3meUufbhgPXwMzXdLJyesId70hD6qkXc20zTiX4rb
 HLoB1/73TJYVkqNE3I9eruBEDiexPhHPxOj7X5VI3KNGOKhILCFuBfxLqn7zr8GzDvss5xvJ
 0QFpSW0eeAbmSSW/yKgjWFLw==
X-IronPort-AV: E=Sophos;i="5.90,199,1643691600"; 
   d="scan'208";a="66782867"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Lw5vOVVgcTwhfZT3fb6b0DyP6S15CHFY6Sif/wY1dnHsLaVrTwopi4AYCnJkzQ/QrBgdms8Y7zmIJa/o1Sw7EqUbhxkqbzmh1xB9ly51ptzVivD1otIVOD+kcAjokgiekEwjaczCHkpZIDD8Gpm5k6zoMMQtJPjMwRSmqARO9E4JOnOAPtHVxxENfhCcaZ3dgJkpRqo9yUeqFApLBg6IuuZnZOcR2nD0xcC5k14KBB2ergx6yeZ8TmAfUWxW3gsN7x/fudOw1MOUd72DRPPViJE5/wo4Vtrsg6gdTAqZSlqb7rV/9TTyCT+3FhYsy9SHw/GKxZr9vLcG3Kyk01dqvA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=hrxBIbVIb/zAeJ6Ruwv6Qih6GnNahcgs+8FNm/FBPpM=;
 b=a0LgwaXQ7N4HeYEuTVICGnVp81kbEaTbvxl7IdKhUveRIbFqYXl2Mkl74JuntNapNNWah4L2awdDe3JX9AzcBGvUeautRs4OhJkule1LjRAFEblKu+9FXp3lfP7B4oAFLcOX2R/Qt9oFIi5UmQ970Jgngufno9TJLuxyWPWXvbTIAqIyobmEtEf+2+/qkqnE4EB4cQc/CTCEb70WucAwlMqNTwvohQnIk0wx0rKS8WVQnwSEr/Df1eXmDu+Sy+XGWV0SwVTiLfPpwm/4Qc7bhyIkYCbxiLMrxZu14hBD8Z5cHwOPPFeHvY2RhVq157UqA2P//iViBSOVyKRcxyHsaA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=hrxBIbVIb/zAeJ6Ruwv6Qih6GnNahcgs+8FNm/FBPpM=;
 b=lcl2b9ZEWmWKiAUJ68vht6Izi5RYZpRYq3rWxIHt4m4w9Lphbc109y68DMO40CFqVLLCrdTqLrq9WH79OCiAwCZ15KL//XQZpc1gOGfl1DEZAhLIXHSrcygq0sfLNnuXgfD1AQm2PZR8dW28xdTyhbtGJPpKZoBGtmuP02iADtQ=
From: Edwin Torok <edvin.torok@citrix.com>
To: Martin Harvey <martin.harvey@citrix.com>
CC: "paul@xen.org" <paul@xen.org>, "win-pv-devel@lists.xenproject.org"
	<win-pv-devel@lists.xenproject.org>
Subject: Re: [PATCH] Windows PV drivers fail to set up RSS when vCPUs > 8
Thread-Topic: [PATCH] Windows PV drivers fail to set up RSS when vCPUs > 8
Thread-Index: AQHYN7gtXmzUE2l3x0isfDYfnRWjD6zHAReAgAMJTjCAAAtmgA==
Date: Mon, 21 Mar 2022 16:42:14 +0000
Message-ID: <7FDE677E-E1C9-4FC1-931F-1FE102558E17@citrix.com>
References: <20220314152746.1712-1-Martin.Harvey@citrix.com>
 <e81b19af-067b-1a98-7952-f06c101a2418@gmail.com>
 <MW4PR03MB63476907571ADA5598BE441E9A169@MW4PR03MB6347.namprd03.prod.outlook.com>
In-Reply-To: <MW4PR03MB63476907571ADA5598BE441E9A169@MW4PR03MB6347.namprd03.prod.outlook.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3693.40.0.1.81)
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 4a7ea88f-fdb1-40a6-f855-08da0b59c15f
x-ms-traffictypediagnostic: BN6PR03MB2626:EE_
x-microsoft-antispam-prvs: <BN6PR03MB2626CEE083B7E27BDAA2544C9B169@BN6PR03MB2626.namprd03.prod.outlook.com>
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: /BOTPzTslW3Ig3asu0kPxz2LziSduZCKwZueMVPdXCuvYmcWIBvIOSD5i9i2YOEGzNnCLhZzI3jdSdjbcHuk7SvG49mHjHWa+zF4o0T08RmHpUWC8npHUaW25FIhFGSTEXUdrZaEkZ90KvboLmniO4XueojKHi888DFRlPuwqHQJFRXcZgeb6J6HmXxoC7QfHmZr7lMBkQFbwdFulNKjzewLKglN3ehXS/Gl9jfmo4bRcSn80NkhBo+k8oEHFH2hZKoRdB1lgjJvq1B1UaPoRtaZTZEHlbGVt2VNv1hF6HffbP+Y2MDA5KXAle3uXcyWEuTB7DGmfmkOKeUA4qaaMSfbBbtH+tu4z2j8Pjg9EZyUtBpog+eUHuJXz1/XDeJRhZlD2O2px0t7HfU2Nm5otQ934uGwom55j8b6d0T2QEk20XMDY72xjTuS1nBmpLDI2HQY5mRIbburkwqXCI6a7/hbNCC3B0Macnwfouy9SHmTNtd23W5lxgvO0ZAmgnvMz1m1iY7ZbsJzgqW//fRIMyb1ABfKPrQ7mNJ/YaJCHO3moEemtk51j6fvrI9GTCZHk2CwiRlGE9FfDl0E8P9Zkc7LFBX8nWXzgHxBgsSD2XhFuJjvkvRoOqhHbrLorLSZgUwymQzUntn0LeOE2sFGbBtyebZ/uQ2e1QKgbS697xxAERBWf9/o6aT3PiGNjVvZtmtTnKrljNXhxS4usAmSByX0vx/Wfvsec2561dfDjFGpQGU2Tg+pz5p5H38UvmAn
x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SJ0PR03MB5888.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(64756008)(6862004)(4326008)(66446008)(66476007)(66556008)(30864003)(53546011)(66946007)(8676002)(82960400001)(6506007)(38100700002)(91956017)(76116006)(316002)(122000001)(71200400001)(86362001)(33656002)(8936002)(38070700005)(2616005)(6636002)(37006003)(54906003)(6512007)(5660300002)(36756003)(6486002)(2906002)(83380400001)(508600001)(186003)(26005)(45980500001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?G+UNLsMezB/Jomw/HqVY3/2mDsWAy9Bkyv8srAR3RRsxYnGQFsiIm69dV418?=
 =?us-ascii?Q?b73u9px8hPN0ZS77BINQMbitx7kbq+o1o+VnQztMRNLhRZyeljSU+QZ0qVUB?=
 =?us-ascii?Q?UV222R/s/QlGc2Jm0P1naS8tju4P4aMmhTv2rySmlcf/jZNkxMRyCe+ZVz5n?=
 =?us-ascii?Q?CcVd2fg98JtW/x4KdG+KZ3d3V8aSctrDLoHB7c/+V7ebCsEQ+P8WkA2TDvtf?=
 =?us-ascii?Q?9QgaGYbZxRBg/WMpxYdUmRgdPzgg5R0JwL9iH+Y7OV908lMJ2xoszhMS/Rci?=
 =?us-ascii?Q?/X0RkmuPktFGZoEKvO7Io9lFJPjW8d2sv5Liskd8/tWnSuEx/V3qEfL8y0cy?=
 =?us-ascii?Q?xdLhk9lYOdGeXErRvuPuabX1P8yar+1iaTOcuqTbrTaT7Vrl0oUToPilo4Kg?=
 =?us-ascii?Q?OJEyexO/LA+2+koXzwh1WEZL9lQROrokC88cqsNh33raMQkaol+5BWny+dEd?=
 =?us-ascii?Q?GSIsu86UKv2GEMOOgbri8vXHcP3fJ4nPrydx+KF0gzuso59Y32RcEWGNhqvm?=
 =?us-ascii?Q?wvChK4XgW3uV9f0lIns1A50LT/cDOgJ1QtOUWFz5zWSjadwJlwrLHn5hQJFH?=
 =?us-ascii?Q?QgggE8Cgjq20li/Ax61Jzc1hlic93FCY+BYa+JEXhzjON6kIH5I1Gr96oKEA?=
 =?us-ascii?Q?89HqUMfyPPR8TzYd7CPiKzY3gEVg0rhP1zIXnwt18MOGL+mEvg8Qvc7hpiU0?=
 =?us-ascii?Q?xZTD5N1dNn+JUzMXYQwaHJNocfTF1/H3rrdVief5JiXW3Vzyaus1cHHyqdst?=
 =?us-ascii?Q?vi+WneOBa42y474CL+3KGjR6GH/J7g6dMAXOzVOaMJUdt7wabT0VlvExRKJp?=
 =?us-ascii?Q?qxEPbRoV6en+0ovltuyKLkoY6th59HBFtRpOGuCmrHy4mnLDU5LUG3sTLUNn?=
 =?us-ascii?Q?HCm9/VqW8QDGcDue1hte7oBmhU6h0Fbxn8LL2CrmKMMytfDaeWr6AEt8qe42?=
 =?us-ascii?Q?CNChf9Vi7qsPpp2FoaRa+u2yFEHpamfjNgmW3cCTaMfurMlUfWB8eiQ5UmUW?=
 =?us-ascii?Q?8zWjIOYjPyvZlkTue6RCeVIIfSkYcV9ykvn46RbYDtDHuq5RKmUqc6jYYfxC?=
 =?us-ascii?Q?5wKVMWZU9r5yVN78aeGWPAOja+wdgCg6JAmRaCypgL57iFKgNBw3Wmolm7sN?=
 =?us-ascii?Q?n5yORGOlw2Uv3Ayd8D0tfR/idYv1bNfV4+29l3Y2MYOekuVrzMxEEkWEAojQ?=
 =?us-ascii?Q?qLbpzqZyKh9qNEyofKm7cwKfZ/2Zv5/vQbXX9q5SQM7K3yPyJjqsacWVNh1L?=
 =?us-ascii?Q?Dn5BREL4OPCPYYuqcAJY+vEG3KftQOUAri0eQQQ4kWOMGAgib1f/1t3lP6ue?=
 =?us-ascii?Q?sDbkxPqfpKUjkJ05H2MY234MDpnk5qQtIm1BYbXT4gVmGu71v4TISDwf4K3U?=
 =?us-ascii?Q?9ZsIOYZUz3j14dQlPUD/etXu+nSbobf/UHIVu5PvsUv/7X0ccBXLzdpirlCQ?=
 =?us-ascii?Q?eLaFYqgMx/HWrCWVeG/mYjURzrwjoNDnPZWyhNUKV5SO6U+rtz04jg=3D=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-ID: <0D8942FAD1AA9341825D5229BDCB3798@namprd03.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SJ0PR03MB5888.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4a7ea88f-fdb1-40a6-f855-08da0b59c15f
X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Mar 2022 16:42:14.9683
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: Zmx7mJeCYnr+fZMhJ07Twypv1vPJyyykfGizVCoM4+bQt6Eic5A+Jsc7NAAc9IcdiztN5YEaozsfgOr1xTEbaA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR03MB2626
X-OriginatorOrg: citrix.com



> On 21 Mar 2022, at 16:06, Martin Harvey <martin.harvey@citrix.com> wrote:
>=20
>=20
>=20
> -----Original Message-----
> From: win-pv-devel <win-pv-devel-bounces@lists.xenproject.org> On Behalf =
Of Durrant, Paul
> Sent: 19 March 2022 17:39
> To: win-pv-devel@lists.xenproject.org
> Subject: Re: [PATCH] Windows PV drivers fail to set up RSS when vCPUs > 8
>=20
> [CAUTION - EXTERNAL EMAIL] DO NOT reply, click links, or open attachments=
 unless you have verified the sender and know the content is safe.
>=20
>=20
>> I think we need a bit more in the commit comment. What is the nature of =
the failure... and does XENNET advertise more than 8 queues, so will the si=
tuation ever arise? Linux certainly tops out at 8 queues.
>=20
> I don't believe that XenNet ever advertises more than 8 queues, but that'=
s not quite the same as supporting more than 8 vCPU's
>=20
> Perhaps something in the comment like: "Mapping between Queues and VPU's =
fails for more than 8 VCPU's because the base of the indirection is always =
considered to be zero, and the mapping is always performed on a direct vCPU=
 number basis"
>=20
> Would that adequately summarise the problem? (It's not easy to explain su=
ccinctly in English!)

Here is another attempt at explaining it, perhaps you could put your one se=
ntence in the commit title, and the one below in the commit body?

The driver only supports at most 8 queues, however Windows can decide to as=
sign vCPU numbers starting from a non-zero offset.=20
E.g. vCPU 8,9,10,11 could get assigned to a device if you have more than on=
e NIC. The total number of vCPUs used by a single device is still less than=
 8, but the vCPU indexes themselves can be greater than 8.
The code previously incorrectly assumed that individual vCPU indexes cannot=
 exceed 8, however a 1:1 mapping between vCPU indexes and queues seems to o=
nly exist when using a single NIC.
[Full example below]

>=20
> MH.
>=20
> This summary helpfully provided by Edvin Torok.
>=20
>=20
> On a VM with >8 vCPUs RSS might not work because the driver fails to set =
up the indirection table.
>=20
> This causes the VM to only be able to reach 12.4Gbit/s with 'iperf3 -P 8'=
, instead of 16-18Gbit/s with a working RSS setup:
>=20
> This can be easily reproduced if you give a VM 32 vCPUs and create 3 netw=
ork interfaces. Windows will assign 0-3 to one network interface, 4-7 to ne=
xt, and will try 8-12 I think for next but the driver rejects that:
>=20
> PS C:\Program Files\CItrix\XenTools\Diagnostics> Get-NetAdapterRSS
>=20
>=20
> Name                                            : Ethernet 5
> InterfaceDescription                            : XenServer PV Network De=
vice #2
> Enabled                                         : True
> NumberOfReceiveQueues                           : 8
> Profile                                         : NUMAStatic
> BaseProcessor: [Group:Number]                   : 0:0
> MaxProcessor: [Group:Number]                    : 0:31
> MaxProcessors                                   : 4
> RssProcessorArray: [Group:Number/NUMA Distance] : 0:0/0  0:1/0  0:2/0  0:=
3/0  0:4/0  0:5/0  0:6/0  0:7/0
>                                                  0:8/0  0:9/0  0:10/0  0:=
11/0  0:12/0  0:13/0  0:14/0  0:15/0
>                                                  0:16/0  0:17/0  0:18/0  =
0:19/0  0:20/0  0:21/0  0:22/0  0:23/0
>                                                  0:24/0  0:25/0  0:26/0  =
0:27/0  0:28/0  0:29/0  0:30/0  0:31/0
> IndirectionTable: [Group:Number]                :
>=20
> Name                                            : Ethernet 4
> InterfaceDescription                            : XenServer PV Network De=
vice #1
> Enabled                                         : True
> NumberOfReceiveQueues                           : 8
> Profile                                         : NUMAStatic
> BaseProcessor: [Group:Number]                   : 0:0
> MaxProcessor: [Group:Number]                    : 0:31
> MaxProcessors                                   : 4
> RssProcessorArray: [Group:Number/NUMA Distance] : 0:0/0  0:1/0  0:2/0  0:=
3/0  0:4/0  0:5/0  0:6/0  0:7/0
>                                                  0:8/0  0:9/0  0:10/0  0:=
11/0  0:12/0  0:13/0  0:14/0  0:15/0
>                                                  0:16/0  0:17/0  0:18/0  =
0:19/0  0:20/0  0:21/0  0:22/0  0:23/0
>                                                  0:24/0  0:25/0  0:26/0  =
0:27/0  0:28/0  0:29/0  0:30/0  0:31/0
> IndirectionTable: [Group:Number]                : 0:4   0:5     0:6     0=
:7     0:4     0:5     0:6     0:7
>                                                  0:4   0:5     0:6     0:=
7     0:4     0:5     0:6     0:7
>                                                  0:4   0:5     0:6     0:=
7     0:4     0:5     0:6     0:7
>                                                  0:4   0:5     0:6     0:=
7     0:4     0:5     0:6     0:7
>                                                  0:4   0:5     0:6     0:=
7     0:4     0:5     0:6     0:7
>                                                  0:4   0:5     0:6     0:=
7     0:4     0:5     0:6     0:7
>                                                  0:4   0:5     0:6     0:=
7     0:4     0:5     0:6     0:7
>                                                  0:4   0:5     0:6     0:=
7     0:4     0:5     0:6     0:7
>                                                  0:4   0:5     0:6     0:=
7     0:4     0:5     0:6     0:7
>                                                  0:4   0:5     0:6     0:=
7     0:4     0:5     0:6     0:7
>                                                  0:4   0:5     0:6     0:=
7     0:4     0:5     0:6     0:7
>                                                  0:4   0:5     0:6     0:=
7     0:4     0:5     0:6     0:7
>                                                  0:4   0:5     0:6     0:=
7     0:4     0:5     0:6     0:7
>                                                  0:4   0:5     0:6     0:=
7     0:4     0:5     0:6     0:7
>                                                  0:4   0:5     0:6     0:=
7     0:4     0:5     0:6     0:7
>                                                  0:4   0:5     0:6     0:=
7     0:4     0:5     0:6     0:7
>=20
> Name                                            : Ethernet 3
> InterfaceDescription                            : XenServer PV Network De=
vice #0
> Enabled                                         : True
> NumberOfReceiveQueues                           : 8
> Profile                                         : NUMAStatic
> BaseProcessor: [Group:Number]                   : 0:0
> MaxProcessor: [Group:Number]                    : 0:31
> MaxProcessors                                   : 4
> RssProcessorArray: [Group:Number/NUMA Distance] : 0:0/0  0:1/0  0:2/0  0:=
3/0  0:4/0  0:5/0  0:6/0  0:7/0
>                                                  0:8/0  0:9/0  0:10/0  0:=
11/0  0:12/0  0:13/0  0:14/0  0:15/0
>                                                  0:16/0  0:17/0  0:18/0  =
0:19/0  0:20/0  0:21/0  0:22/0  0:23/0
>                                                  0:24/0  0:25/0  0:26/0  =
0:27/0  0:28/0  0:29/0  0:30/0  0:31/0
> IndirectionTable: [Group:Number]                : 0:0   0:1     0:2     0=
:3     0:0     0:1     0:2     0:3
>                                                  0:0   0:1     0:2     0:=
3     0:0     0:1     0:2     0:3
>                                                  0:0   0:1     0:2     0:=
3     0:0     0:1     0:2     0:3
>                                                  0:0   0:1     0:2     0:=
3     0:0     0:1     0:2     0:3
>                                                  0:0   0:1     0:2     0:=
3     0:0     0:1     0:2     0:3
>                                                  0:0   0:1     0:2     0:=
3     0:0     0:1     0:2     0:3
>                                                  0:0   0:1     0:2     0:=
3     0:0     0:1     0:2     0:3
>                                                  0:0   0:1     0:2     0:=
3     0:0     0:1     0:2     0:3
>                                                  0:0   0:1     0:2     0:=
3     0:0     0:1     0:2     0:3
>                                                  0:0   0:1     0:2     0:=
3     0:0     0:1     0:2     0:3
>                                                  0:0   0:1     0:2     0:=
3     0:0     0:1     0:2     0:3
>                                                  0:0   0:1     0:2     0:=
3     0:0     0:1     0:2     0:3
>                                                  0:0   0:1     0:2     0:=
3     0:0     0:1     0:2     0:3
>                                                  0:0   0:1     0:2     0:=
3     0:0     0:1     0:2     0:3
>                                                  0:0   0:1     0:2     0:=
3     0:0     0:1     0:2     0:3
>                                                  0:0   0:1     0:2     0:=
3     0:0     0:1     0:2     0:3
> There is a builtin hardcoded limit of 8 queues, which is fine, but that s=
hould be completely unrelated to CPU numbers! (the total number of CPUs ass=
igned to a NIC should be <=3D8, sure).
>=20
> Potential code causing issue in xenvif receiver.c:
>=20
> for (Index =3D 0; Index < Size; Index++) {
>          QueueMapping[Index] =3D KeGetProcessorIndexFromNumber(&Processor=
Mapping[Index]);
>          if (QueueMapping[Index] >=3D NumQueues)
>              goto fail2;
>      }
>=20
> (there is also a problem that the code assumes that group number is alway=
s 0. For now that is true, but might change if we implement vNUMA in the fu=
ture).
>=20
> Jun  2 09:54:57 prost qemu-dm-41[30818]: 30818@1622627697.450233:xen_plat=
form_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
> Jun  2 09:54:57 prost qemu-dm-41[30818]: 30818@1622627697.450320:xen_plat=
form_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
> Jun  2 09:54:57 prost qemu-dm-41[30818]: 30818@1622627697.452097:xen_plat=
form_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
> Jun  2 09:54:57 prost qemu-dm-41[30818]: 30818@1622627697.452180:xen_plat=
form_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
> Jun  2 09:56:14 prost qemu-dm-41[30818]: 30818@1622627774.374713:xen_plat=
form_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
> Jun  2 09:56:14 prost qemu-dm-41[30818]: 30818@1622627774.374798:xen_plat=
form_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
> Jun  2 09:56:14 prost qemu-dm-41[30818]: 30818@1622627774.377121:xen_plat=
form_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
> Jun  2 09:56:14 prost qemu-dm-41[30818]: 30818@1622627774.377203:xen_plat=
form_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
> Jun  2 09:59:00 prost qemu-dm-42[1106]: 1106@1622627940.672941:xen_platfo=
rm_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
> Jun  2 09:59:00 prost qemu-dm-42[1106]: 1106@1622627940.673058:xen_platfo=
rm_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
> Jun  2 09:59:00 prost qemu-dm-42[1106]: 1106@1622627940.675891:xen_platfo=
rm_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
> Jun  2 09:59:00 prost qemu-dm-42[1106]: 1106@1622627940.675993:xen_platfo=
rm_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
> Jun  2 09:59:39 prost qemu-dm-43[4074]: 4074@1622627979.363892:xen_platfo=
rm_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
> Jun  2 09:59:39 prost qemu-dm-43[4074]: 4074@1622627979.364008:xen_platfo=
rm_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
> Jun  2 09:59:39 prost qemu-dm-43[4074]: 4074@1622627979.365861:xen_platfo=
rm_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
> Jun  2 09:59:39 prost qemu-dm-43[4074]: 4074@1622627979.365949:xen_platfo=
rm_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
> Jun  2 10:04:41 prost qemu-dm-45[9705]: 9705@1622628281.935871:xen_platfo=
rm_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
> Jun  2 10:04:41 prost qemu-dm-45[9705]: 9705@1622628281.935965:xen_platfo=
rm_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
> Jun  2 10:04:41 prost qemu-dm-45[9705]: 9705@1622628281.937849:xen_platfo=
rm_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
> Jun  2 10:04:41 prost qemu-dm-45[9705]: 9705@1622628281.937918:xen_platfo=
rm_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
> Jun  2 10:05:00 prost qemu-dm-46[11484]: 11484@1622628300.973487:xen_plat=
form_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
> Jun  2 10:05:00 prost qemu-dm-46[11484]: 11484@1622628300.973588:xen_plat=
form_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
> Jun  2 10:05:00 prost qemu-dm-46[11484]: 11484@1622628300.976554:xen_plat=
form_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
> Jun  2 10:05:00 prost qemu-dm-46[11484]: 11484@1622628300.976650:xen_plat=
form_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
> Jun  2 10:22:54 prost qemu-dm-49[21901]: 21901@1622629374.720769:xen_plat=
form_log xen platform: xenvif|PdoGetInterfaceGuid: fail1 (c0000034)
> Jun  2 10:22:55 prost qemu-dm-49[21901]: 21901@1622629375.194122:xen_plat=
form_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
> Jun  2 10:22:55 prost qemu-dm-49[21901]: 21901@1622629375.194231:xen_plat=
form_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
> Jun  2 10:22:55 prost qemu-dm-49[21901]: 21901@1622629375.196726:xen_plat=
form_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
> Jun  2 10:22:55 prost qemu-dm-49[21901]: 21901@1622629375.196825:xen_plat=
form_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
> Jun  2 10:23:38 prost qemu-dm-50[24509]: 24509@1622629418.530046:xen_plat=
form_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
> Jun  2 10:23:38 prost qemu-dm-50[24509]: 24509@1622629418.530115:xen_plat=
form_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
> Jun  2 10:23:38 prost qemu-dm-50[24509]: 24509@1622629418.531811:xen_plat=
form_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
> Jun  2 10:23:38 prost qemu-dm-50[24509]: 24509@1622629418.531888:xen_plat=
form_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
> Jun  2 10:30:28 prost qemu-dm-51[28530]: 28530@1622629828.510968:xen_plat=
form_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
> Jun  2 10:30:28 prost qemu-dm-51[28530]: 28530@1622629828.511050:xen_plat=
form_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
> Jun  2 10:30:28 prost qemu-dm-51[28530]: 28530@1622629828.513570:xen_plat=
form_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
> Jun  2 10:30:28 prost qemu-dm-51[28530]: 28530@1622629828.513691:xen_plat=
form_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
> Jun  2 10:45:43 prost qemu-dm-52[2889]: 2889@1622630743.573791:xen_platfo=
rm_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
> Jun  2 10:45:43 prost qemu-dm-52[2889]: 2889@1622630743.573904:xen_platfo=
rm_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
> Jun  2 10:45:43 prost qemu-dm-52[2889]: 2889@1622630743.576188:xen_platfo=
rm_log xen platform: xenvif|ReceiverUpdateHashMapping: fail2
> Jun  2 10:45:43 prost qemu-dm-52[2889]: 2889@1622630743.576298:xen_platfo=
rm_log xen platform: xenvif|ReceiverUpdateHashMapping: fail1 (c000000d)
>=20
> I tested with both Win10 and Windows Server 2016, with various CPU topolo=
gies (e.g. 12 vCPUs, all on one socket shows same issue once windows starts=
 assigning CPUs>8)
>=20
> A workaround is to set MaxProcessorNumber to 7, though obviously this wil=
l limit scalability since vCPUs > 8 won't be used even if you have multiple=
 VIFs:
>=20
>=20
>=20



From win-pv-devel-bounces@lists.xenproject.org Tue Mar 22 13:43:58 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Tue, 22 Mar 2022 13:43:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.293475.498577 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nWenl-000053-HL; Tue, 22 Mar 2022 13:43:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 293475.498577; Tue, 22 Mar 2022 13:43:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nWenl-00004v-Dw; Tue, 22 Mar 2022 13:43:57 +0000
Received: by outflank-mailman (input) for mailman id 293475;
 Tue, 22 Mar 2022 13:43:56 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UN57=UB=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1nWenk-0008WV-AI
 for win-pv-devel@lists.xenproject.org; Tue, 22 Mar 2022 13:43:56 +0000
Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com
 [2a00:1450:4864:20::331])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1c7363e8-a9e6-11ec-a405-831a346695d4;
 Tue, 22 Mar 2022 14:43:52 +0100 (CET)
Received: by mail-wm1-x331.google.com with SMTP id
 r190-20020a1c2bc7000000b0038a1013241dso1778190wmr.1
 for <win-pv-devel@lists.xenproject.org>; Tue, 22 Mar 2022 06:43:52 -0700 (PDT)
Received: from [10.139.200.51] ([54.239.6.189])
 by smtp.gmail.com with ESMTPSA id
 t4-20020a05600001c400b00203fb5dcf29sm9778575wrx.40.2022.03.22.06.43.51
 for <win-pv-devel@lists.xenproject.org>
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 22 Mar 2022 06:43:51 -0700 (PDT)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1c7363e8-a9e6-11ec-a405-831a346695d4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=message-id:date:mime-version:user-agent:reply-to:subject
         :content-language:to:references:from:in-reply-to
         :content-transfer-encoding;
        bh=fJe4V6VKSJ/AxZfnh1BLuRrjrA8zMFToLckcpizjuPU=;
        b=ljhyrpYhyc/JV+Z3wYFIdiI00k++W3kYzHDhNkYsWlUBgx3z3nr+orZoDIUuaYgqy1
         lG6etRkGwv1NIzaMb8KXuj5md5jUiDDJdhGj8RAyNAgpJiQt0w4McUmV89c6hE7ivWVe
         4DCX2wFh/Myw53a9VXd0WYvsXtwFh4yijWQsij/FExs3Tcme3qwBbGohOLkkzfX58Dfi
         IKISd2KgpK327eb49Ixtdx2VfGQXxpUc++AW8bcIJAMP+zlKmKopMawEQjeTLtfl6qXS
         zhwKn7Ln+ZRaCkoRZKh2hLouYVre12/jBXkhMKTI35RBKihKzlJJ+gf7Whbi5KaHVTJN
         6F1w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:message-id:date:mime-version:user-agent:reply-to
         :subject:content-language:to:references:from:in-reply-to
         :content-transfer-encoding;
        bh=fJe4V6VKSJ/AxZfnh1BLuRrjrA8zMFToLckcpizjuPU=;
        b=AJCw33yXBoaSSuAig/XAYBAMv7SFtFM0PDixNm4QqA9VIYuaCc5y8xv4+/9OpnQ7EH
         18q5wfDud9uKGtRAVX283qlUVAjUgOMHJQCLQl8WbGxdGQd8cIRtd3bMTM+3yQfSOc7r
         SRCLq4OZYB1wmD/9h+w1NoCY0gogpM/pUv+8cJwWSJkgDy8/77Z9QoLNQ6RS/W3gBCPL
         rdUqusZG+DtiH6ITrSnSOKAx3GJzD2VG0EZKNkZvZArm+aCyYYCGkoTPUL2hDx+3pln+
         HjbuOu8oVRyMnjl25vZtoWcT5kfisntCiIMyCOivg+lWiqS/yX/vw9Jx2jOSihwbxtHv
         McdA==
X-Gm-Message-State: AOAM532LUFHCWwB5lK4xYp81ugYyZBzlqwKlPIPoncWtlS5XPJDBgDnI
	N1LXP1YqjbLmJtWL+A8Cb652qwpT7wSLWw==
X-Google-Smtp-Source: ABdhPJyT3LktTzqmGaUI8AwK8dA04azZc1WMewwJTZ2oswi5Wkp2E70KSAbldxoLop5niUB3kEJh+w==
X-Received: by 2002:a7b:c778:0:b0:38c:9064:89fc with SMTP id x24-20020a7bc778000000b0038c906489fcmr3864525wmk.175.1647956631760;
        Tue, 22 Mar 2022 06:43:51 -0700 (PDT)
Message-ID: <da370f45-4ded-6f4e-7c0b-60917d0afa05@gmail.com>
Date: Tue, 22 Mar 2022 14:43:50 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.7.0
Reply-To: paul@xen.org
Subject: Re: [PATCH] Improve Xenfilt power and default IRP handling.
Content-Language: en-US
To: win-pv-devel@lists.xenproject.org
References: <20220314170141.1565-1-Martin.Harvey@citrix.com>
From: "Durrant, Paul" <xadimgnik@gmail.com>
In-Reply-To: <20220314170141.1565-1-Martin.Harvey@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 14/03/2022 18:01, Martin Harvey wrote:

This is a very large patch given absolutely no explanation. It really 
needs a *much* more comprehensive commit comment... even be broken down 
into multiple incremental patches, e.g. maybe doing FDO and PDO separately.

> Signed-off-by: Martin Harvey <martin.harvey@citrix.com>
> ---
>   src/common/util.h |   8 +
>   src/xenfilt/fdo.c | 785 ++++++++++++---------------------------------
>   src/xenfilt/pdo.c | 790 ++++++++++++----------------------------------
>   3 files changed, 411 insertions(+), 1172 deletions(-)
> 
> diff --git a/src/common/util.h b/src/common/util.h
> index 36a36dd..fb9b9b7 100644
> --- a/src/common/util.h
> +++ b/src/common/util.h
> @@ -36,6 +36,14 @@
>   
>   #include "assert.h"
>   
> +typedef enum _XEN_IRP_STRATEGY {
> +    XEN_IRP_PASSDOWN_TRANSPARENT,
> +    XEN_IRP_HANDLE_INDISPATCH_AND_COMPLETE,
> +    XEN_IRP_HANDLE_INDISPATCH_PASSDOWN,
> +    XEN_IRP_HANDLE_INCOMPLETION,

I'm not keen on these names. What is 'incompletion'? I.e. your use of 
'_' or lack thereof seems rather arbitrary.

> +    XEN_IRP_STRATEGY_MAX
> +} XEN_IRP_STRATEGY, *PXEN_IRP_STRATEGY;
> +
>   #define	P2ROUNDUP(_x, _a)   \
>           (-(-(_x) & -(_a)))
>   
> diff --git a/src/xenfilt/fdo.c b/src/xenfilt/fdo.c
> index 63fa7b3..5d43d01 100644
> --- a/src/xenfilt/fdo.c
> +++ b/src/xenfilt/fdo.c
> @@ -59,11 +59,6 @@ struct _XENFILT_FDO {
>       PDEVICE_OBJECT                  PhysicalDeviceObject;
>       CHAR                            Name[MAXNAMELEN];
>   
> -    PXENFILT_THREAD                 SystemPowerThread;
> -    PIRP                            SystemPowerIrp;
> -    PXENFILT_THREAD                 DevicePowerThread;
> -    PIRP                            DevicePowerIrp;
> -
>       MUTEX                           Mutex;
>       LIST_ENTRY                      List;
>       ULONG                           References;
> @@ -1004,6 +999,26 @@ fail1:
>       return status;
>   }
>   
> +static NTSTATUS
> +FdoDispatchDefault(
> +    IN  PXENFILT_FDO    Fdo,
> +    IN  PIRP            Irp
> +    )
> +{
> +    NTSTATUS status;
> +    IoSkipCurrentIrpStackLocation(Irp);
> +    status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
> +    if (!NT_SUCCESS(status))
> +        goto fail1;
> +
> +    return status;
> +
> +fail1:
> +    Error("fail1 (%08x)\n", status);
> +    return status;
> +}
> +
> +
>   static NTSTATUS
>   FdoDispatchPnp(
>       IN  PXENFILT_FDO    Fdo,
> @@ -1055,524 +1070,135 @@ FdoDispatchPnp(
>           break;
>   
>       default:
> -        status = IoAcquireRemoveLock(&Fdo->Dx->RemoveLock, Irp);
> -        if (!NT_SUCCESS(status))
> -            goto fail1;
> -
> -        IoSkipCurrentIrpStackLocation(Irp);
> -
> -        status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
> -        IoReleaseRemoveLock(&Fdo->Dx->RemoveLock, Irp);
> +        status = FdoDispatchDefault(Fdo, Irp);
>           break;
>       }
>   
> -    return status;
> -
> -fail1:
> -    Error("fail1 (%08x)\n", status);
> -
> -    Irp->IoStatus.Status = status;
> -    IoCompleteRequest(Irp, IO_NO_INCREMENT);
> -
> -    return status;
> -}
> -
> -static NTSTATUS
> -FdoSetDevicePowerUp(
> -    IN  PXENFILT_FDO    Fdo,
> -    IN  PIRP            Irp
> -    )
> -{
> -    PIO_STACK_LOCATION  StackLocation;
> -    DEVICE_POWER_STATE  DeviceState;
> -    POWER_STATE         PowerState;
> -    NTSTATUS            status;
> -
> -    StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
> -
> -    ASSERT3U(DeviceState, <,  __FdoGetDevicePowerState(Fdo));
> -
> -    status = FdoForwardIrpSynchronously(Fdo, Irp);
> -    if (!NT_SUCCESS(status))
> -        goto done;
> -
> -    Trace("%s: %s -> %s\n",
> -          __FdoGetName(Fdo),
> -          DevicePowerStateName(__FdoGetDevicePowerState(Fdo)),
> -          DevicePowerStateName(DeviceState));
> -
> -    PowerState.DeviceState = DeviceState;
> -    PoSetPowerState(Fdo->Dx->DeviceObject,
> -                    DevicePowerState,
> -                    PowerState);
> -
> -    __FdoSetDevicePowerState(Fdo, DeviceState);
> -
> -done:
> -    IoCompleteRequest(Irp, IO_NO_INCREMENT);
> -
> -    return status;
> -}
> -
> -static NTSTATUS
> -FdoSetDevicePowerDown(
> -    IN  PXENFILT_FDO    Fdo,
> -    IN  PIRP            Irp
> -    )
> -{
> -    PIO_STACK_LOCATION  StackLocation;
> -    DEVICE_POWER_STATE  DeviceState;
> -    POWER_STATE         PowerState;
> -    NTSTATUS            status;
> -
> -    StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
> -
> -    ASSERT3U(DeviceState, >,  __FdoGetDevicePowerState(Fdo));
> -
> -    Trace("%s: %s -> %s\n",
> -          __FdoGetName(Fdo),
> -          DevicePowerStateName(__FdoGetDevicePowerState(Fdo)),
> -          DevicePowerStateName(DeviceState));
> -
> -    PowerState.DeviceState = DeviceState;
> -    PoSetPowerState(Fdo->Dx->DeviceObject,
> -                    DevicePowerState,
> -                    PowerState);
> -
> -    __FdoSetDevicePowerState(Fdo, DeviceState);
> -
> -    status = FdoForwardIrpSynchronously(Fdo, Irp);
> -    IoCompleteRequest(Irp, IO_NO_INCREMENT);
> -
> -    return status;
> -}
> -
> -static NTSTATUS
> -FdoSetDevicePower(
> -    IN  PXENFILT_FDO    Fdo,
> -    IN  PIRP            Irp
> -    )
> -{
> -    PIO_STACK_LOCATION  StackLocation;
> -    DEVICE_POWER_STATE  DeviceState;
> -    POWER_ACTION        PowerAction;
> -    NTSTATUS            status;
> -
> -    StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
> -    PowerAction = StackLocation->Parameters.Power.ShutdownType;
> -
> -    Trace("%s: ====> (%s:%s)\n",
> -          __FdoGetName(Fdo),
> -          DevicePowerStateName(DeviceState),
> -          PowerActionName(PowerAction));
> -
> -    if (DeviceState == __FdoGetDevicePowerState(Fdo)) {
> -        status = FdoForwardIrpSynchronously(Fdo, Irp);
> -        IoCompleteRequest(Irp, IO_NO_INCREMENT);
> -
> -        goto done;
> -    }
> -
> -    status = (DeviceState < __FdoGetDevicePowerState(Fdo)) ?
> -             FdoSetDevicePowerUp(Fdo, Irp) :
> -             FdoSetDevicePowerDown(Fdo, Irp);
> -
> -done:
> -    Trace("%s: <==== (%s:%s)(%08x)\n",
> -          __FdoGetName(Fdo),
> -          DevicePowerStateName(DeviceState),
> -          PowerActionName(PowerAction),
> -          status);
> -    return status;
> -}
> -
> -static NTSTATUS
> -FdoSetSystemPowerUp(
> -    IN  PXENFILT_FDO    Fdo,
> -    IN  PIRP            Irp
> -    )
> -{
> -    PIO_STACK_LOCATION  StackLocation;
> -    SYSTEM_POWER_STATE  SystemState;
> -    NTSTATUS            status;
> -
> -    StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -    SystemState = StackLocation->Parameters.Power.State.SystemState;
> -
> -    ASSERT3U(SystemState, <,  __FdoGetSystemPowerState(Fdo));
> -
> -    status = FdoForwardIrpSynchronously(Fdo, Irp);
> -    if (!NT_SUCCESS(status))
> -        goto done;
> -
> -    Trace("%s: %s -> %s\n",
> -          __FdoGetName(Fdo),
> -          SystemPowerStateName(__FdoGetSystemPowerState(Fdo)),
> -          SystemPowerStateName(SystemState));
> -
> -    __FdoSetSystemPowerState(Fdo, SystemState);
> -
> -done:
> -    IoCompleteRequest(Irp, IO_NO_INCREMENT);
> -
> -    return status;
> -}
> -
> -static NTSTATUS
> -FdoSetSystemPowerDown(
> -    IN  PXENFILT_FDO     Fdo,
> -    IN  PIRP            Irp
> -    )
> -{
> -    PIO_STACK_LOCATION  StackLocation;
> -    SYSTEM_POWER_STATE  SystemState;
> -    NTSTATUS            status;
> -
> -    StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -    SystemState = StackLocation->Parameters.Power.State.SystemState;
> -
> -    ASSERT3U(SystemState, >,  __FdoGetSystemPowerState(Fdo));
> -
> -    Trace("%s: %s -> %s\n",
> -          __FdoGetName(Fdo),
> -          SystemPowerStateName(__FdoGetSystemPowerState(Fdo)),
> -          SystemPowerStateName(SystemState));
> -
> -    __FdoSetSystemPowerState(Fdo, SystemState);
> -
> -    status = FdoForwardIrpSynchronously(Fdo, Irp);
> -    IoCompleteRequest(Irp, IO_NO_INCREMENT);
> -
> -    return status;
> -}
> -
> -static NTSTATUS
> -FdoSetSystemPower(
> -    IN  PXENFILT_FDO    Fdo,
> -    IN  PIRP            Irp
> -    )
> -{
> -    PIO_STACK_LOCATION  StackLocation;
> -    SYSTEM_POWER_STATE  SystemState;
> -    POWER_ACTION        PowerAction;
> -    NTSTATUS            status;
> -
> -    StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -    SystemState = StackLocation->Parameters.Power.State.SystemState;
> -    PowerAction = StackLocation->Parameters.Power.ShutdownType;
> -
> -    Trace("%s: ====> (%s:%s)\n",
> -          __FdoGetName(Fdo),
> -          SystemPowerStateName(SystemState),
> -          PowerActionName(PowerAction));
> -
> -    if (SystemState == __FdoGetSystemPowerState(Fdo)) {
> -        status = FdoForwardIrpSynchronously(Fdo, Irp);
> -        IoCompleteRequest(Irp, IO_NO_INCREMENT);
> -
> -        goto done;
> -    }
> -
> -    status = (SystemState < __FdoGetSystemPowerState(Fdo)) ?
> -             FdoSetSystemPowerUp(Fdo, Irp) :
> -             FdoSetSystemPowerDown(Fdo, Irp);
> -
> -done:
> -    Trace("%s: <==== (%s:%s)(%08x)\n",
> -          __FdoGetName(Fdo),
> -          SystemPowerStateName(SystemState),
> -          PowerActionName(PowerAction),
> -          status);
> -    return status;
> -}
> -
> -static NTSTATUS
> -FdoQueryDevicePowerUp(
> -    IN  PXENFILT_FDO    Fdo,
> -    IN  PIRP            Irp
> -    )
> -{
> -    PIO_STACK_LOCATION  StackLocation;
> -    DEVICE_POWER_STATE  DeviceState;
> -    NTSTATUS            status;
> -
> -    StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
> -
> -    ASSERT3U(DeviceState, <,  __FdoGetDevicePowerState(Fdo));
> -
> -    status = FdoForwardIrpSynchronously(Fdo, Irp);
> -
> -    IoCompleteRequest(Irp, IO_NO_INCREMENT);
> -
>       return status;
>   }
>   
>   static NTSTATUS
> -FdoQueryDevicePowerDown(
> +FdoDispatchPowerImmediate(

Is there a non-immediate variant? I can't see it.

>       IN  PXENFILT_FDO    Fdo,
>       IN  PIRP            Irp
>       )
>   {
> -    PIO_STACK_LOCATION  StackLocation;
> -    DEVICE_POWER_STATE  DeviceState;
> -    NTSTATUS            status;
> -
> -    StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
> +    NTSTATUS status = STATUS_INTERNAL_ERROR;
> +    PIO_STACK_LOCATION StackLocation = IoGetCurrentIrpStackLocation(Irp);
> +    UCHAR MinorFunction = StackLocation->MinorFunction;
>   
> -    ASSERT3U(DeviceState, >,  __FdoGetDevicePowerState(Fdo));
> -
> -    status = FdoForwardIrpSynchronously(Fdo, Irp);
> -    IoCompleteRequest(Irp, IO_NO_INCREMENT);
> -
> -    return status;
> -}
> -
> -static NTSTATUS
> -FdoQueryDevicePower(
> -    IN  PXENFILT_FDO    Fdo,
> -    IN  PIRP            Irp
> -    )
> -{
> -    PIO_STACK_LOCATION  StackLocation;
> -    DEVICE_POWER_STATE  DeviceState;
> -    POWER_ACTION        PowerAction;
> -    NTSTATUS            status;
> +    if (MinorFunction != IRP_MN_SET_POWER)
> +        goto fail1;
>   
> -    StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
> -    PowerAction = StackLocation->Parameters.Power.ShutdownType;
> +    POWER_STATE_TYPE PowerType = StackLocation->Parameters.Power.Type;
>   
> -    Trace("%s: ====> (%s:%s)\n",
> -          __FdoGetName(Fdo),
> -          DevicePowerStateName(DeviceState),
> -          PowerActionName(PowerAction));
> +    switch (PowerType) {
> +    case DevicePowerState:
> +        POWER_STATE PowerState;
> +        DEVICE_POWER_STATE DeviceState = StackLocation->Parameters.Power.State.DeviceState;
> +        if (DeviceState >= __FdoGetDevicePowerState(Fdo))
> +            goto fail3;
>   
> -    if (DeviceState == __FdoGetDevicePowerState(Fdo)) {
> -        status = FdoForwardIrpSynchronously(Fdo, Irp);
> -        IoCompleteRequest(Irp, IO_NO_INCREMENT);
> +        __FdoSetDevicePowerState(Fdo, DeviceState);
> +        PowerState.DeviceState = DeviceState;
> +        PoSetPowerState(__FdoGetDeviceObject(Fdo),
> +                        DevicePowerState,
> +                        PowerState);
> +    break;
> +    case SystemPowerState:
> +        SYSTEM_POWER_STATE SystemState = StackLocation->Parameters.Power.State.SystemState;
> +        if (SystemState >= __FdoGetSystemPowerState(Fdo))
> +            goto fail3;
>   
> -        goto done;
> +        __FdoSetSystemPowerState(Fdo, SystemState);
> +    break;
> +    default:
> +        goto fail2;
>       }
>   
> -    status = (DeviceState < __FdoGetDevicePowerState(Fdo)) ?
> -             FdoQueryDevicePowerUp(Fdo, Irp) :
> -             FdoQueryDevicePowerDown(Fdo, Irp);
> -
> -done:
> -    Trace("%s: <==== (%s:%s)(%08x)\n",
> -          __FdoGetName(Fdo),
> -          DevicePowerStateName(DeviceState),
> -          PowerActionName(PowerAction),
> -          status);
> -    return status;
> -}
> -
> -static NTSTATUS
> -FdoQuerySystemPowerUp(
> -    IN  PXENFILT_FDO     Fdo,
> -    IN  PIRP            Irp
> -    )
> -{
> -    PIO_STACK_LOCATION  StackLocation;
> -    SYSTEM_POWER_STATE  SystemState;
> -    NTSTATUS            status;
> -
> -    StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -    SystemState = StackLocation->Parameters.Power.State.SystemState;
> -
> -    ASSERT3U(SystemState, <,  __FdoGetSystemPowerState(Fdo));
> -
> -    status = FdoForwardIrpSynchronously(Fdo, Irp);
> -
> -    IoCompleteRequest(Irp, IO_NO_INCREMENT);
> +    return STATUS_SUCCESS;
>   
> +fail3:
> +    Error("fail3 (%08x)\n", status);

We don't need to spit status out at every fail level.

> +fail2:
> +    Error("fail2 (%08x)\n", status);
> +fail1:
> +    Error("fail1 (%08x)\n", status);
>       return status;
>   }
>   
> -static NTSTATUS
> -FdoQuerySystemPowerDown(
> +static VOID
> +FdoCompletionPower(
>       IN  PXENFILT_FDO    Fdo,
>       IN  PIRP            Irp
> -    )
> +)
>   {
> -    PIO_STACK_LOCATION  StackLocation;
> -    SYSTEM_POWER_STATE  SystemState;
> -    NTSTATUS            status;
> +    PIO_STACK_LOCATION StackLocation = IoGetCurrentIrpStackLocation(Irp);
> +    UCHAR MinorFunction = StackLocation->MinorFunction;
>   
> -    StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -    SystemState = StackLocation->Parameters.Power.State.SystemState;
> -
> -    ASSERT3U(SystemState, >,  __FdoGetSystemPowerState(Fdo));
> -
> -    status = FdoForwardIrpSynchronously(Fdo, Irp);
> -    IoCompleteRequest(Irp, IO_NO_INCREMENT);
> -
> -    return status;
> -}
> +    if (MinorFunction != IRP_MN_SET_POWER)
> +        goto fail1;
>   
> -static NTSTATUS
> -FdoQuerySystemPower(
> -    IN  PXENFILT_FDO    Fdo,
> -    IN  PIRP            Irp
> -    )
> -{
> -    PIO_STACK_LOCATION  StackLocation;
> -    SYSTEM_POWER_STATE  SystemState;
> -    POWER_ACTION        PowerAction;
> -    NTSTATUS            status;
> +    POWER_STATE_TYPE PowerType = StackLocation->Parameters.Power.Type;
>   
> -    StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -    SystemState = StackLocation->Parameters.Power.State.SystemState;
> -    PowerAction = StackLocation->Parameters.Power.ShutdownType;
> +    switch (PowerType) {
> +    case DevicePowerState:
> +        POWER_STATE PowerState;
> +        DEVICE_POWER_STATE DeviceState = StackLocation->Parameters.Power.State.DeviceState;
> +        if (DeviceState <= __FdoGetDevicePowerState(Fdo))
> +            goto fail3;
>   
> -    Trace("%s: ====> (%s:%s)\n",
> -          __FdoGetName(Fdo),
> -          SystemPowerStateName(SystemState),
> -          PowerActionName(PowerAction));
> +        PowerState.DeviceState = DeviceState;
> +        PoSetPowerState(__FdoGetDeviceObject(Fdo),
> +                        DevicePowerState,
> +                        PowerState);
> +        __FdoSetDevicePowerState(Fdo, DeviceState);
>   
> -    if (SystemState == __FdoGetSystemPowerState(Fdo)) {
> -        status = FdoForwardIrpSynchronously(Fdo, Irp);
> -        IoCompleteRequest(Irp, IO_NO_INCREMENT);
> +    break;
> +    case SystemPowerState:
> +        SYSTEM_POWER_STATE SystemState = StackLocation->Parameters.Power.State.SystemState;
> +        if (SystemState <= __FdoGetSystemPowerState(Fdo))
> +            goto fail3;
>   
> -        goto done;
> +        __FdoSetSystemPowerState(Fdo, SystemState);
> +    break;
> +    default:
> +        goto fail2;
>       }
>   
> -    status = (SystemState < __FdoGetSystemPowerState(Fdo)) ?
> -             FdoQuerySystemPowerUp(Fdo, Irp) :
> -             FdoQuerySystemPowerDown(Fdo, Irp);
> -
> -done:
> -    Trace("%s: <==== (%s:%s)(%08x)\n",
> -          __FdoGetName(Fdo),
> -          SystemPowerStateName(SystemState),
> -          PowerActionName(PowerAction),
> -          status);
> +    return;
>   
> -    return status;
> +fail3:
> +    Error("fail3\n");
> +fail2:
> +    Error("fail2\n");
> +fail1:
> +    Error("fail1");
>   }
>   
> +__drv_functionClass(IO_COMPLETION_ROUTINE)
> +__drv_sameIRQL
>   static NTSTATUS
> -FdoDevicePower(
> -    IN  PXENFILT_THREAD Self,
> +FdoDispatchPowerCompletionV2(

Ick. 'V2'? Really? What is V2 about it?

> +    IN  PDEVICE_OBJECT  DeviceObject,
> +    IN  PIRP            Irp,
>       IN  PVOID           Context
>       )
>   {
>       PXENFILT_FDO        Fdo = Context;
> -    PKEVENT             Event;
> -
> -    Event = ThreadGetEvent(Self);
> -
> -    for (;;) {
> -        PIRP                Irp;
> -        PIO_STACK_LOCATION  StackLocation;
> -        UCHAR               MinorFunction;
> -
> -        if (Fdo->DevicePowerIrp == NULL) {
> -            (VOID) KeWaitForSingleObject(Event,
> -                                         Executive,
> -                                         KernelMode,
> -                                         FALSE,
> -                                         NULL);
> -            KeClearEvent(Event);
> -        }
> -
> -        if (ThreadIsAlerted(Self))
> -            break;
> -
> -        Irp = Fdo->DevicePowerIrp;
> -
> -        if (Irp == NULL)
> -            continue;
>   
> -        Fdo->DevicePowerIrp = NULL;
> -        KeMemoryBarrier();
> -
> -        StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -        MinorFunction = StackLocation->MinorFunction;
> -
> -        switch (StackLocation->MinorFunction) {
> -        case IRP_MN_SET_POWER:
> -            (VOID) FdoSetDevicePower(Fdo, Irp);
> -            break;
> -
> -        case IRP_MN_QUERY_POWER:
> -            (VOID) FdoQueryDevicePower(Fdo, Irp);
> -            break;
> +    UNREFERENCED_PARAMETER(DeviceObject);
>   
> -        default:
> -            ASSERT(FALSE);
> -            break;
> -        }
> +    if (Irp->PendingReturned)
> +        IoMarkIrpPending(Irp);
>   
> -        IoReleaseRemoveLock(&Fdo->Dx->RemoveLock, Irp);
> -    }
> +    //Assumption don't need STATUS_MORE_PROCESSING_REQUIRED.
> +    FdoCompletionPower(Fdo, Irp);
>   
> +    IoReleaseRemoveLock(&Fdo->Dx->RemoveLock, Irp);
>       return STATUS_SUCCESS;
>   }
>   
> -static NTSTATUS
> -FdoSystemPower(
> -    IN  PXENFILT_THREAD Self,
> -    IN  PVOID           Context
> -    )
> -{
> -    PXENFILT_FDO        Fdo = Context;
> -    PKEVENT             Event;
> -
> -    Event = ThreadGetEvent(Self);
> -
> -    for (;;) {
> -        PIRP                Irp;
> -        PIO_STACK_LOCATION  StackLocation;
> -        UCHAR               MinorFunction;
> -
> -        if (Fdo->SystemPowerIrp == NULL) {
> -            (VOID) KeWaitForSingleObject(Event,
> -                                         Executive,
> -                                         KernelMode,
> -                                         FALSE,
> -                                         NULL);
> -            KeClearEvent(Event);
> -        }
> -
> -        if (ThreadIsAlerted(Self))
> -            break;
> -
> -        Irp = Fdo->SystemPowerIrp;
> -
> -        if (Irp == NULL)
> -            continue;
> -
> -        Fdo->SystemPowerIrp = NULL;
> -        KeMemoryBarrier();
> -
> -        StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -        MinorFunction = StackLocation->MinorFunction;
> -
> -        switch (StackLocation->MinorFunction) {
> -        case IRP_MN_SET_POWER:
> -            (VOID) FdoSetSystemPower(Fdo, Irp);
> -            break;
> -
> -        case IRP_MN_QUERY_POWER:
> -            (VOID) FdoQuerySystemPower(Fdo, Irp);
> -            break;
> -
> -        default:
> -            ASSERT(FALSE);
> -            break;
> -        }
> -
> -        IoReleaseRemoveLock(&Fdo->Dx->RemoveLock, Irp);
> -    }
> -
> -    return STATUS_SUCCESS;
> -}
>   
>   static NTSTATUS
>   FdoDispatchPower(
> @@ -1582,104 +1208,125 @@ FdoDispatchPower(
>   {
>       PIO_STACK_LOCATION  StackLocation;
>       UCHAR               MinorFunction;
> -    POWER_STATE_TYPE    PowerType;
>       NTSTATUS            status;
> -
> -    status = IoAcquireRemoveLock(&Fdo->Dx->RemoveLock, Irp);
> -    if (!NT_SUCCESS(status))
> -        goto fail1;
> +    XEN_IRP_STRATEGY    strategy;
> +    BOOLEAN HaveRemoveLock = FALSE;
>   
>       StackLocation = IoGetCurrentIrpStackLocation(Irp);
>       MinorFunction = StackLocation->MinorFunction;
>   
> -    if (MinorFunction != IRP_MN_QUERY_POWER &&
> -        MinorFunction != IRP_MN_SET_POWER) {
> -        IoSkipCurrentIrpStackLocation(Irp);
> -
> -        status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
> -        IoReleaseRemoveLock(&Fdo->Dx->RemoveLock, Irp);
> -
> -        goto done;
> -    }
> -
> -    PowerType = StackLocation->Parameters.Power.Type;
> -
> -    Trace("%s: ====> (%02x:%s)\n",
> -          __FdoGetName(Fdo),
> -          MinorFunction,
> -          PowerMinorFunctionName(MinorFunction));
> -
> -    switch (PowerType) {
> -    case DevicePowerState:
> -        IoMarkIrpPending(Irp);
> -
> -        ASSERT3P(Fdo->DevicePowerIrp, ==, NULL);
> -        Fdo->DevicePowerIrp = Irp;
> -        KeMemoryBarrier();
> -
> -        ThreadWake(Fdo->DevicePowerThread);
> -
> -        status = STATUS_PENDING;
> +    //Decide how we're going to handle this IRP.
> +    switch(MinorFunction) {
> +    case IRP_MN_QUERY_POWER:
> +        strategy = XEN_IRP_PASSDOWN_TRANSPARENT; //Dont fail, don't need to check queries.
>           break;
> -
> -    case SystemPowerState:
> -        IoMarkIrpPending(Irp);
> -
> -        ASSERT3P(Fdo->SystemPowerIrp, ==, NULL);
> -        Fdo->SystemPowerIrp = Irp;
> -        KeMemoryBarrier();
> -
> -        ThreadWake(Fdo->SystemPowerThread);
> -
> -        status = STATUS_PENDING;
> +    case IRP_MN_SET_POWER:
> +        POWER_STATE_TYPE PowerType = StackLocation->Parameters.Power.Type;
> +
> +        switch (PowerType) {
> +        case DevicePowerState:
> +            DEVICE_POWER_STATE DeviceState = StackLocation->Parameters.Power.State.DeviceState;
> +
> +            status = IoAcquireRemoveLock(&Fdo->Dx->RemoveLock, Irp);
> +            if (!NT_SUCCESS(status))
> +                goto fail1;
> +
> +            //If fails, then pending removal or already removed,
> +            //freshly initialized (or not). Consider status to return...

Can we have more complete english in comments so they are clear? Also a 
space after the '//' please.

> +            HaveRemoveLock = TRUE;
> +
> +            if (DeviceState == __FdoGetDevicePowerState(Fdo)) {
> +                strategy = XEN_IRP_PASSDOWN_TRANSPARENT;

No need for braces in this sequence (and similar elsewhere).

> +            } else if (DeviceState < __FdoGetDevicePowerState(Fdo)) {
> +                strategy = XEN_IRP_HANDLE_INDISPATCH_PASSDOWN;
> +            } else {
> +                strategy = XEN_IRP_HANDLE_INCOMPLETION;
> +            }
> +        case SystemPowerState:
> +            SYSTEM_POWER_STATE SystemState = StackLocation->Parameters.Power.State.SystemState;
> +
> +            status = IoAcquireRemoveLock(&Fdo->Dx->RemoveLock, Irp);
> +            if (!NT_SUCCESS(status))
> +                goto fail1;
> +
> +            //If fails, then pending removal or already removed,
> +            //freshly initialized (or not). Consider status to return...
> +            HaveRemoveLock = TRUE;
> +
> +            if (SystemState == __FdoGetSystemPowerState(Fdo)) {
> +                strategy = XEN_IRP_PASSDOWN_TRANSPARENT;
> +            } else if (SystemState < __FdoGetSystemPowerState(Fdo)) {
> +                strategy = XEN_IRP_HANDLE_INDISPATCH_PASSDOWN;
> +            } else {
> +                strategy = XEN_IRP_HANDLE_INCOMPLETION;
> +            }
>           break;
> -
> +        default:
> +            strategy = XEN_IRP_PASSDOWN_TRANSPARENT;
> +            break;
> +        }
>       default:
> -        IoSkipCurrentIrpStackLocation(Irp);
> -
> -        status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
> -        IoReleaseRemoveLock(&Fdo->Dx->RemoveLock, Irp);
> +        strategy = XEN_IRP_PASSDOWN_TRANSPARENT;
>           break;
>       }
>   
> -    Trace("%s: <==== (%02x:%s) (%08x)\n",
> -          __FdoGetName(Fdo),
> -          MinorFunction,
> -          PowerMinorFunctionName(MinorFunction),
> -          status);
> -
> -done:
> -    return status;
> -
> -fail1:
> -    Error("fail1 (%08x)\n", status);
> +    switch(strategy) {
> +        case XEN_IRP_HANDLE_INDISPATCH_PASSDOWN:
> +            status = FdoDispatchPowerImmediate(Fdo, Irp);
> +            if (!NT_SUCCESS(status))
> +                goto fail2;
> +            //Fallthru.

Is that a valid annotation format?

> +        case XEN_IRP_PASSDOWN_TRANSPARENT:
> +        default:
> +            IoSkipCurrentIrpStackLocation(Irp);
> +            status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
> +            if (!NT_SUCCESS(status))
> +                goto fail3;
>   
> -    Irp->IoStatus.Status = status;
> -    IoCompleteRequest(Irp, IO_NO_INCREMENT);
> +            if (HaveRemoveLock)
> +                IoReleaseRemoveLock(&Fdo->Dx->RemoveLock, Irp);
> +        break;
> +        case XEN_IRP_HANDLE_INCOMPLETION:
> +            IoMarkIrpPending(Irp);
> +            IoCopyCurrentIrpStackLocationToNext(Irp);
> +            IoSetCompletionRoutine(Irp,
> +                                FdoDispatchPowerCompletionV2,
> +                                Fdo,
> +                                TRUE,
> +                                TRUE,
> +                                TRUE);
> +            status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
> +            if (!NT_SUCCESS(status))
> +                goto fail4;
> +            status = STATUS_PENDING; //Must return STATUS_PENDING after IoMarkIrpPending, whatever happens next.
> +        break;
> +    }
>   
>       return status;
> -}
> -
> -static NTSTATUS
> -FdoDispatchDefault(
> -    IN  PXENFILT_FDO    Fdo,
> -    IN  PIRP            Irp
> -    )
> -{
> -    NTSTATUS            status;
> -
> -    status = IoAcquireRemoveLock(&Fdo->Dx->RemoveLock, Irp);
> -    if (!NT_SUCCESS(status))
> -        goto fail1;
>   
> -    IoSkipCurrentIrpStackLocation(Irp);
> -
> -    status = IoCallDriver(Fdo->LowerDeviceObject, Irp);
> -    IoReleaseRemoveLock(&Fdo->Dx->RemoveLock, Irp);
> +fail4:
> +    //Error from lower driver, have remove lock, and completion routine.
> +    //Comp routine shd be called on error to drop remove lock.
> +    Error("fail4 (%08x)\n", status);
> +    status = STATUS_PENDING; //Must return STATUS_PENDING after IoMarkIrpPending, whatever happens next.
> +    return status;

The whole point of fail labels is that they undo things in sequence and 
*only* fail1 returns.

>   
> +fail3:
> +    //Error from lower driver, have remove lock, called down.
> +    Error("fail3 (%08x)\n", status);
> +    if (HaveRemoveLock)
> +        IoReleaseRemoveLock(&Fdo->Dx->RemoveLock, Irp);
>       return status;
>   
> +fail2:
> +    //Immediate error, have remove lock, not called down.
> +    Error("fail2 (%08x)\n", status);
> +    if (HaveRemoveLock)
> +        IoReleaseRemoveLock(&Fdo->Dx->RemoveLock, Irp);
> +    //Fallthru.
> +
>   fail1:
> +    //Immediate error, no remove lock, not called down.
>       Error("fail1 (%08x)\n", status);
>   
>       Irp->IoStatus.Status = status;
> @@ -1775,14 +1422,6 @@ FdoCreate(
>       Fdo->LowerDeviceObject = LowerDeviceObject;
>       Fdo->Type = Type;
>   
> -    status = ThreadCreate(FdoSystemPower, Fdo, &Fdo->SystemPowerThread);
> -    if (!NT_SUCCESS(status))
> -        goto fail4;
> -
> -    status = ThreadCreate(FdoDevicePower, Fdo, &Fdo->DevicePowerThread);
> -    if (!NT_SUCCESS(status))
> -        goto fail5;
> -
>       status = __FdoSetDeviceID(Fdo);
>       if (!NT_SUCCESS(status))
>           goto fail6;
> @@ -1822,20 +1461,6 @@ fail7:
>   fail6:
>       Error("fail6\n");
>   
> -    ThreadAlert(Fdo->DevicePowerThread);
> -    ThreadJoin(Fdo->DevicePowerThread);
> -    Fdo->DevicePowerThread = NULL;
> -
> -fail5:
> -    Error("fail5\n");
> -
> -    ThreadAlert(Fdo->SystemPowerThread);
> -    ThreadJoin(Fdo->SystemPowerThread);
> -    Fdo->SystemPowerThread = NULL;
> -
> -fail4:
> -    Error("fail4\n");
> -
>       Fdo->Type = XENFILT_EMULATED_OBJECT_TYPE_UNKNOWN;
>       Fdo->PhysicalDeviceObject = NULL;
>       Fdo->LowerDeviceObject = NULL;
> @@ -1891,14 +1516,6 @@ FdoDestroy(
>       __FdoClearInstanceID(Fdo);
>       __FdoClearDeviceID(Fdo);
>   
> -    ThreadAlert(Fdo->DevicePowerThread);
> -    ThreadJoin(Fdo->DevicePowerThread);
> -    Fdo->DevicePowerThread = NULL;
> -
> -    ThreadAlert(Fdo->SystemPowerThread);
> -    ThreadJoin(Fdo->SystemPowerThread);
> -    Fdo->SystemPowerThread = NULL;
> -
>       Fdo->Type = XENFILT_EMULATED_OBJECT_TYPE_UNKNOWN;
>       Fdo->LowerDeviceObject = NULL;
>       Fdo->PhysicalDeviceObject = NULL;
> diff --git a/src/xenfilt/pdo.c b/src/xenfilt/pdo.c
> index 741c2f3..ee2a049 100644
> --- a/src/xenfilt/pdo.c
> +++ b/src/xenfilt/pdo.c
> @@ -56,11 +56,6 @@ struct _XENFILT_PDO {
>       PDEVICE_OBJECT                  PhysicalDeviceObject;
>       CHAR                            Name[MAXNAMELEN];
>   
> -    PXENFILT_THREAD                 SystemPowerThread;
> -    PIRP                            SystemPowerIrp;
> -    PXENFILT_THREAD                 DevicePowerThread;
> -    PIRP                            DevicePowerIrp;
> -
>       PXENFILT_FDO                    Fdo;
>       BOOLEAN                         Missing;
>       const CHAR                      *Reason;
> @@ -1101,6 +1096,26 @@ PdoEject(
>       return status;
>   }
>   
> +static NTSTATUS
> +PdoDispatchDefault(
> +    IN  PXENFILT_PDO    Pdo,
> +    IN  PIRP            Irp
> +    )
> +{
> +    NTSTATUS status;
> +    IoSkipCurrentIrpStackLocation(Irp);
> +    status = IoCallDriver(Pdo->LowerDeviceObject, Irp);
> +    if (!NT_SUCCESS(status))
> +        goto fail1;
> +
> +    return status;
> +
> +fail1:
> +    Error("fail1 (%08x)\n", status);
> +    return status;
> +}
> +
> +
>   static NTSTATUS
>   PdoDispatchPnp(
>       IN  PXENFILT_PDO    Pdo,
> @@ -1164,528 +1179,136 @@ PdoDispatchPnp(
>           break;
>   
>       default:
> -        status = IoAcquireRemoveLock(&Pdo->Dx->RemoveLock, Irp);
> -        if (!NT_SUCCESS(status))
> -            goto fail1;
> -
> -        IoSkipCurrentIrpStackLocation(Irp);
> -
> -        status = IoCallDriver(Pdo->LowerDeviceObject, Irp);
> -        IoReleaseRemoveLock(&Pdo->Dx->RemoveLock, Irp);
> +        status = PdoDispatchDefault(Pdo, Irp);
>           break;
>       }
>   
> -    return status;
> -
> -fail1:
> -    Error("fail1 (%08x)\n", status);
> -
> -    Irp->IoStatus.Status = status;
> -    IoCompleteRequest(Irp, IO_NO_INCREMENT);
> -
> -    return status;
> -}
> -
> -static NTSTATUS
> -PdoSetDevicePowerUp(
> -    IN  PXENFILT_PDO    Pdo,
> -    IN  PIRP            Irp
> -    )
> -{
> -    PIO_STACK_LOCATION  StackLocation;
> -    DEVICE_POWER_STATE  DeviceState;
> -    POWER_STATE         PowerState;
> -    NTSTATUS            status;
> -
> -    StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
> -
> -    ASSERT3U(DeviceState, <,  __PdoGetDevicePowerState(Pdo));
> -
> -    status = PdoForwardIrpSynchronously(Pdo, Irp);
> -    if (!NT_SUCCESS(status))
> -        goto done;
> -
> -    Trace("%s: %s -> %s\n",
> -          __PdoGetName(Pdo),
> -          DevicePowerStateName(__PdoGetDevicePowerState(Pdo)),
> -          DevicePowerStateName(DeviceState));
> -
> -    PowerState.DeviceState = DeviceState;
> -    PoSetPowerState(__PdoGetDeviceObject(Pdo),
> -                    DevicePowerState,
> -                    PowerState);
> -
> -    __PdoSetDevicePowerState(Pdo, DeviceState);
> -
> -done:
> -    Irp->IoStatus.Status = status;
> -    IoCompleteRequest(Irp, IO_NO_INCREMENT);
> -
>       return status;
>   }
>   
> -static NTSTATUS
> -PdoSetDevicePowerDown(
> -    IN  PXENFILT_PDO    Pdo,
> -    IN  PIRP            Irp
> -    )
> -{
> -    PIO_STACK_LOCATION  StackLocation;
> -    DEVICE_POWER_STATE  DeviceState;
> -    POWER_STATE         PowerState;
> -    NTSTATUS            status;
> -
> -    StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
> -
> -    ASSERT3U(DeviceState, >,  __PdoGetDevicePowerState(Pdo));
> -
> -    Trace("%s: %s -> %s\n",
> -          __PdoGetName(Pdo),
> -          DevicePowerStateName(__PdoGetDevicePowerState(Pdo)),
> -          DevicePowerStateName(DeviceState));
> -
> -    PowerState.DeviceState = DeviceState;
> -    PoSetPowerState(__PdoGetDeviceObject(Pdo),
> -                    DevicePowerState,
> -                    PowerState);
> -
> -    __PdoSetDevicePowerState(Pdo, DeviceState);
> -
> -    status = PdoForwardIrpSynchronously(Pdo, Irp);
> -    IoCompleteRequest(Irp, IO_NO_INCREMENT);
> -
> -    return status;
> -}
>   
>   static NTSTATUS
> -PdoSetDevicePower(
> +PdoDispatchPowerImmediate(
>       IN  PXENFILT_PDO    Pdo,
>       IN  PIRP            Irp
>       )
>   {
> -    PIO_STACK_LOCATION  StackLocation;
> -    DEVICE_POWER_STATE  DeviceState;
> -    POWER_ACTION        PowerAction;
> -    NTSTATUS            status;
> +    NTSTATUS status = STATUS_INTERNAL_ERROR;
> +    PIO_STACK_LOCATION StackLocation = IoGetCurrentIrpStackLocation(Irp);
> +    UCHAR MinorFunction = StackLocation->MinorFunction;
>   
> -    StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
> -    PowerAction = StackLocation->Parameters.Power.ShutdownType;
> -
> -    Trace("%s: ====> (%s:%s)\n",
> -          __PdoGetName(Pdo),
> -          DevicePowerStateName(DeviceState),
> -          PowerActionName(PowerAction));
> -
> -    if (DeviceState == __PdoGetDevicePowerState(Pdo)) {
> -        status = PdoForwardIrpSynchronously(Pdo, Irp);
> -        IoCompleteRequest(Irp, IO_NO_INCREMENT);
> -
> -        goto done;
> -    }
> -
> -    status = (DeviceState < __PdoGetDevicePowerState(Pdo)) ?
> -             PdoSetDevicePowerUp(Pdo, Irp) :
> -             PdoSetDevicePowerDown(Pdo, Irp);
> -
> -done:
> -    Trace("%s: <==== (%s:%s)(%08x)\n",
> -          __PdoGetName(Pdo),
> -          DevicePowerStateName(DeviceState),
> -          PowerActionName(PowerAction),
> -          status);
> -    return status;
> -}
> -
> -static NTSTATUS
> -PdoSetSystemPowerUp(
> -    IN  PXENFILT_PDO    Pdo,
> -    IN  PIRP            Irp
> -    )
> -{
> -    PIO_STACK_LOCATION  StackLocation;
> -    SYSTEM_POWER_STATE  SystemState;
> -    NTSTATUS            status;
> -
> -    StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -    SystemState = StackLocation->Parameters.Power.State.SystemState;
> -
> -    ASSERT3U(SystemState, <,  __PdoGetSystemPowerState(Pdo));
> -
> -    status = PdoForwardIrpSynchronously(Pdo, Irp);
> -    if (!NT_SUCCESS(status))
> -        goto done;
> -
> -    Trace("%s: %s -> %s\n",
> -          __PdoGetName(Pdo),
> -          SystemPowerStateName(__PdoGetSystemPowerState(Pdo)),
> -          SystemPowerStateName(SystemState));
> -
> -    __PdoSetSystemPowerState(Pdo, SystemState);
> -
> -done:
> -    Irp->IoStatus.Status = status;
> -    IoCompleteRequest(Irp, IO_NO_INCREMENT);
> -
> -    return status;
> -}
> -
> -static NTSTATUS
> -PdoSetSystemPowerDown(
> -    IN  PXENFILT_PDO     Pdo,
> -    IN  PIRP            Irp
> -    )
> -{
> -    PIO_STACK_LOCATION  StackLocation;
> -    SYSTEM_POWER_STATE  SystemState;
> -    NTSTATUS            status;
> -
> -    StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -    SystemState = StackLocation->Parameters.Power.State.SystemState;
> -
> -    ASSERT3U(SystemState, >,  __PdoGetSystemPowerState(Pdo));
> -
> -    Trace("%s: %s -> %s\n",
> -          __PdoGetName(Pdo),
> -          SystemPowerStateName(__PdoGetSystemPowerState(Pdo)),
> -          SystemPowerStateName(SystemState));
> -
> -    __PdoSetSystemPowerState(Pdo, SystemState);
> -
> -    status = PdoForwardIrpSynchronously(Pdo, Irp);
> -    IoCompleteRequest(Irp, IO_NO_INCREMENT);
> -
> -    return status;
> -}
> -
> -static NTSTATUS
> -PdoSetSystemPower(
> -    IN  PXENFILT_PDO    Pdo,
> -    IN  PIRP            Irp
> -    )
> -{
> -    PIO_STACK_LOCATION  StackLocation;
> -    SYSTEM_POWER_STATE  SystemState;
> -    POWER_ACTION        PowerAction;
> -    NTSTATUS            status;
> -
> -    StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -    SystemState = StackLocation->Parameters.Power.State.SystemState;
> -    PowerAction = StackLocation->Parameters.Power.ShutdownType;
> -
> -    Trace("%s: ====> (%s:%s)\n",
> -          __PdoGetName(Pdo),
> -          SystemPowerStateName(SystemState),
> -          PowerActionName(PowerAction));
> -
> -    if (SystemState == __PdoGetSystemPowerState(Pdo)) {
> -        status = PdoForwardIrpSynchronously(Pdo, Irp);
> -        IoCompleteRequest(Irp, IO_NO_INCREMENT);
> -
> -        goto done;
> -    }
> -
> -    status = (SystemState < __PdoGetSystemPowerState(Pdo)) ?
> -             PdoSetSystemPowerUp(Pdo, Irp) :
> -             PdoSetSystemPowerDown(Pdo, Irp);
> -
> -done:
> -    Trace("%s: <==== (%s:%s)(%08x)\n",
> -          __PdoGetName(Pdo),
> -          SystemPowerStateName(SystemState),
> -          PowerActionName(PowerAction),
> -          status);
> -    return status;
> -}
> -
> -static NTSTATUS
> -PdoQueryDevicePowerUp(
> -    IN  PXENFILT_PDO    Pdo,
> -    IN  PIRP            Irp
> -    )
> -{
> -    PIO_STACK_LOCATION  StackLocation;
> -    DEVICE_POWER_STATE  DeviceState;
> -    NTSTATUS            status;
> -
> -    StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
> -
> -    ASSERT3U(DeviceState, <,  __PdoGetDevicePowerState(Pdo));
> -
> -    status = PdoForwardIrpSynchronously(Pdo, Irp);
> -
> -    Irp->IoStatus.Status = status;
> -    IoCompleteRequest(Irp, IO_NO_INCREMENT);
> -
> -    return status;
> -}
> -
> -static NTSTATUS
> -PdoQueryDevicePowerDown(
> -    IN  PXENFILT_PDO    Pdo,
> -    IN  PIRP            Irp
> -    )
> -{
> -    PIO_STACK_LOCATION  StackLocation;
> -    DEVICE_POWER_STATE  DeviceState;
> -    NTSTATUS            status;
> -
> -    StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
> -
> -    ASSERT3U(DeviceState, >,  __PdoGetDevicePowerState(Pdo));
> -
> -    status = PdoForwardIrpSynchronously(Pdo, Irp);
> -    IoCompleteRequest(Irp, IO_NO_INCREMENT);
> -
> -    return status;
> -}
> -
> -static NTSTATUS
> -PdoQueryDevicePower(
> -    IN  PXENFILT_PDO    Pdo,
> -    IN  PIRP            Irp
> -    )
> -{
> -    PIO_STACK_LOCATION  StackLocation;
> -    DEVICE_POWER_STATE  DeviceState;
> -    POWER_ACTION        PowerAction;
> -    NTSTATUS            status;
> -
> -    StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -    DeviceState = StackLocation->Parameters.Power.State.DeviceState;
> -    PowerAction = StackLocation->Parameters.Power.ShutdownType;
> +    if (MinorFunction != IRP_MN_SET_POWER)
> +        goto fail1;
>   
> -    Trace("%s: ====> (%s:%s)\n",
> -          __PdoGetName(Pdo),
> -          DevicePowerStateName(DeviceState),
> -          PowerActionName(PowerAction));
> +    POWER_STATE_TYPE PowerType = StackLocation->Parameters.Power.Type;
>   
> -    if (DeviceState == __PdoGetDevicePowerState(Pdo)) {
> -        status = PdoForwardIrpSynchronously(Pdo, Irp);
> -        IoCompleteRequest(Irp, IO_NO_INCREMENT);
> +    switch (PowerType) {
> +    case DevicePowerState:
> +        POWER_STATE PowerState;
> +        DEVICE_POWER_STATE DeviceState = StackLocation->Parameters.Power.State.DeviceState;
> +        if (DeviceState >= __PdoGetDevicePowerState(Pdo))
> +            goto fail3;
> +
> +        __PdoSetDevicePowerState(Pdo, DeviceState);
> +        PowerState.DeviceState = DeviceState;
> +        PoSetPowerState(__PdoGetDeviceObject(Pdo),
> +                        DevicePowerState,
> +                        PowerState);
> +    break;
> +    case SystemPowerState:
> +        SYSTEM_POWER_STATE SystemState = StackLocation->Parameters.Power.State.SystemState;
> +        if (SystemState >= __PdoGetSystemPowerState(Pdo))
> +            goto fail3;
>   
> -        goto done;
> +        __PdoSetSystemPowerState(Pdo, SystemState);
> +    break;
> +    default:
> +        goto fail2;
>       }
>   
> -    status = (DeviceState < __PdoGetDevicePowerState(Pdo)) ?
> -             PdoQueryDevicePowerUp(Pdo, Irp) :
> -             PdoQueryDevicePowerDown(Pdo, Irp);
> -
> -done:
> -    Trace("%s: <==== (%s:%s)(%08x)\n",
> -          __PdoGetName(Pdo),
> -          DevicePowerStateName(DeviceState),
> -          PowerActionName(PowerAction),
> -          status);
> -    return status;
> -}
> -
> -static NTSTATUS
> -PdoQuerySystemPowerUp(
> -    IN  PXENFILT_PDO    Pdo,
> -    IN  PIRP            Irp
> -    )
> -{
> -    PIO_STACK_LOCATION  StackLocation;
> -    SYSTEM_POWER_STATE  SystemState;
> -    NTSTATUS            status;
> -
> -    StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -    SystemState = StackLocation->Parameters.Power.State.SystemState;
> -
> -    ASSERT3U(SystemState, <,  __PdoGetSystemPowerState(Pdo));
> -
> -    status = PdoForwardIrpSynchronously(Pdo, Irp);
> -
> -    Irp->IoStatus.Status = status;
> -    IoCompleteRequest(Irp, IO_NO_INCREMENT);
> -
> -    return status;
> -}
> -
> -static NTSTATUS
> -PdoQuerySystemPowerDown(
> -    IN  PXENFILT_PDO    Pdo,
> -    IN  PIRP            Irp
> -    )
> -{
> -    PIO_STACK_LOCATION  StackLocation;
> -    SYSTEM_POWER_STATE  SystemState;
> -    NTSTATUS            status;
> -
> -    StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -    SystemState = StackLocation->Parameters.Power.State.SystemState;
> -
> -    ASSERT3U(SystemState, >,  __PdoGetSystemPowerState(Pdo));
> -
> -    status = PdoForwardIrpSynchronously(Pdo, Irp);
> -    IoCompleteRequest(Irp, IO_NO_INCREMENT);
> +    return STATUS_SUCCESS;
>   
> +fail3:
> +    Error("fail3 (%08x)\n", status);
> +fail2:
> +    Error("fail2 (%08x)\n", status);
> +fail1:
> +    Error("fail1 (%08x)\n", status);
>       return status;
>   }
>   
> -static NTSTATUS
> -PdoQuerySystemPower(
> +static VOID
> +PdoCompletionPower(
>       IN  PXENFILT_PDO    Pdo,
>       IN  PIRP            Irp
> -    )
> +)
>   {
> -    PIO_STACK_LOCATION  StackLocation;
> -    SYSTEM_POWER_STATE  SystemState;
> -    POWER_ACTION        PowerAction;
> -    NTSTATUS            status;
> +    PIO_STACK_LOCATION StackLocation = IoGetCurrentIrpStackLocation(Irp);
> +    UCHAR MinorFunction = StackLocation->MinorFunction;
>   
> -    StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -    SystemState = StackLocation->Parameters.Power.State.SystemState;
> -    PowerAction = StackLocation->Parameters.Power.ShutdownType;
> +    if (MinorFunction != IRP_MN_SET_POWER)
> +        goto fail1;
>   
> -    Trace("%s: ====> (%s:%s)\n",
> -          __PdoGetName(Pdo),
> -          SystemPowerStateName(SystemState),
> -          PowerActionName(PowerAction));
> +    POWER_STATE_TYPE PowerType = StackLocation->Parameters.Power.Type;
>   
> -    if (SystemState == __PdoGetSystemPowerState(Pdo)) {
> -        status = PdoForwardIrpSynchronously(Pdo, Irp);
> -        IoCompleteRequest(Irp, IO_NO_INCREMENT);
> +    switch (PowerType) {
> +    case DevicePowerState:
> +        POWER_STATE PowerState;
> +        DEVICE_POWER_STATE DeviceState = StackLocation->Parameters.Power.State.DeviceState;
> +        if (DeviceState <= __PdoGetDevicePowerState(Pdo))
> +            goto fail3;
> +
> +        PowerState.DeviceState = DeviceState;
> +        PoSetPowerState(__PdoGetDeviceObject(Pdo),
> +                        DevicePowerState,
> +                        PowerState);
> +        __PdoSetDevicePowerState(Pdo, DeviceState);
> +
> +    break;
> +    case SystemPowerState:
> +        SYSTEM_POWER_STATE SystemState = StackLocation->Parameters.Power.State.SystemState;
> +        if (SystemState <= __PdoGetSystemPowerState(Pdo))
> +            goto fail3;
>   
> -        goto done;
> +        __PdoSetSystemPowerState(Pdo, SystemState);
> +    break;
> +    default:
> +        goto fail2;
>       }
>   
> -    status = (SystemState < __PdoGetSystemPowerState(Pdo)) ?
> -             PdoQuerySystemPowerUp(Pdo, Irp) :
> -             PdoQuerySystemPowerDown(Pdo, Irp);
> +    return;
>   
> -done:
> -    Trace("%s: <==== (%s:%s)(%08x)\n",
> -          __PdoGetName(Pdo),
> -          SystemPowerStateName(SystemState),
> -          PowerActionName(PowerAction),
> -          status);
> -
> -    return status;
> +fail3:
> +    Error("fail3\n");
> +fail2:
> +    Error("fail2\n");
> +fail1:
> +    Error("fail1");
>   }
>   
> +__drv_functionClass(IO_COMPLETION_ROUTINE)
> +__drv_sameIRQL
>   static NTSTATUS
> -PdoDevicePower(
> -    IN  PXENFILT_THREAD Self,
> +PdoDispatchPowerCompletionV2(

I assume this means that you largely duped the FDO code. Please please 
find some better names.

> +    IN  PDEVICE_OBJECT  DeviceObject,
> +    IN  PIRP            Irp,
>       IN  PVOID           Context
>       )
>   {
>       PXENFILT_PDO        Pdo = Context;
> -    PKEVENT             Event;
> -
> -    Event = ThreadGetEvent(Self);
> -
> -    for (;;) {
> -        PIRP                Irp;
> -        PIO_STACK_LOCATION  StackLocation;
> -        UCHAR               MinorFunction;
> -
> -        if (Pdo->DevicePowerIrp == NULL) {
> -            (VOID) KeWaitForSingleObject(Event,
> -                                         Executive,
> -                                         KernelMode,
> -                                         FALSE,
> -                                         NULL);
> -            KeClearEvent(Event);
> -        }
> -
> -        if (ThreadIsAlerted(Self))
> -            break;
> -
> -        Irp = Pdo->DevicePowerIrp;
> -
> -        if (Irp == NULL)
> -            continue;
> -
> -        Pdo->DevicePowerIrp = NULL;
> -        KeMemoryBarrier();
> -
> -        StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -        MinorFunction = StackLocation->MinorFunction;
>   
> -        switch (StackLocation->MinorFunction) {
> -        case IRP_MN_SET_POWER:
> -            (VOID) PdoSetDevicePower(Pdo, Irp);
> -            break;
> -
> -        case IRP_MN_QUERY_POWER:
> -            (VOID) PdoQueryDevicePower(Pdo, Irp);
> -            break;
> +    UNREFERENCED_PARAMETER(DeviceObject);
>   
> -        default:
> -            ASSERT(FALSE);
> -            break;
> -        }
> +    if (Irp->PendingReturned)
> +        IoMarkIrpPending(Irp);
>   
> -        IoReleaseRemoveLock(&Pdo->Dx->RemoveLock, Irp);
> -    }
> +    //Assumption don't need STATUS_MORE_PROCESSING_REQUIRED.
> +    PdoCompletionPower(Pdo, Irp);
>   
> +    IoReleaseRemoveLock(&Pdo->Dx->RemoveLock, Irp);
>       return STATUS_SUCCESS;
>   }
>   
> -static NTSTATUS
> -PdoSystemPower(
> -    IN  PXENFILT_THREAD Self,
> -    IN  PVOID           Context
> -    )
> -{
> -    PXENFILT_PDO        Pdo = Context;
> -    PKEVENT             Event;
> -
> -    Event = ThreadGetEvent(Self);
> -
> -    for (;;) {
> -        PIRP                Irp;
> -        PIO_STACK_LOCATION  StackLocation;
> -        UCHAR               MinorFunction;
> -
> -        if (Pdo->SystemPowerIrp == NULL) {
> -            (VOID) KeWaitForSingleObject(Event,
> -                                         Executive,
> -                                         KernelMode,
> -                                         FALSE,
> -                                         NULL);
> -            KeClearEvent(Event);
> -        }
> -
> -        if (ThreadIsAlerted(Self))
> -            break;
> -
> -        Irp = Pdo->SystemPowerIrp;
> -
> -        if (Irp == NULL)
> -            continue;
> -
> -        Pdo->SystemPowerIrp = NULL;
> -        KeMemoryBarrier();
> -
> -        StackLocation = IoGetCurrentIrpStackLocation(Irp);
> -        MinorFunction = StackLocation->MinorFunction;
> -
> -        switch (StackLocation->MinorFunction) {
> -        case IRP_MN_SET_POWER:
> -            (VOID) PdoSetSystemPower(Pdo, Irp);
> -            break;
> -
> -        case IRP_MN_QUERY_POWER:
> -            (VOID) PdoQuerySystemPower(Pdo, Irp);
> -            break;
> -
> -        default:
> -            ASSERT(FALSE);
> -            break;
> -        }
> -
> -        IoReleaseRemoveLock(&Pdo->Dx->RemoveLock, Irp);
> -    }
> -
> -    return STATUS_SUCCESS;
> -}
>   
>   static NTSTATUS
>   PdoDispatchPower(
> @@ -1695,104 +1318,125 @@ PdoDispatchPower(
>   {
>       PIO_STACK_LOCATION  StackLocation;
>       UCHAR               MinorFunction;
> -    POWER_STATE_TYPE    PowerType;
>       NTSTATUS            status;
> -
> -    status = IoAcquireRemoveLock(&Pdo->Dx->RemoveLock, Irp);
> -    if (!NT_SUCCESS(status))
> -        goto fail1;
> +    XEN_IRP_STRATEGY    strategy;
> +    BOOLEAN HaveRemoveLock = FALSE;

Here an elsewhere... can we keep the declarations vertically aligned and 
at a tabstop?

>   
>       StackLocation = IoGetCurrentIrpStackLocation(Irp);
>       MinorFunction = StackLocation->MinorFunction;
>   
> -    if (MinorFunction != IRP_MN_QUERY_POWER &&
> -        MinorFunction != IRP_MN_SET_POWER) {
> -        IoSkipCurrentIrpStackLocation(Irp);
> -
> -        status = IoCallDriver(Pdo->LowerDeviceObject, Irp);
> -        IoReleaseRemoveLock(&Pdo->Dx->RemoveLock, Irp);
> -
> -        goto done;
> -    }
> -
> -    PowerType = StackLocation->Parameters.Power.Type;
> -
> -    Trace("%s: ====> (%02x:%s)\n",
> -          __PdoGetName(Pdo),
> -          MinorFunction,
> -          PowerMinorFunctionName(MinorFunction));
> -
> -    switch (PowerType) {
> -    case DevicePowerState:
> -        IoMarkIrpPending(Irp);
> -
> -        ASSERT3P(Pdo->DevicePowerIrp, ==, NULL);
> -        Pdo->DevicePowerIrp = Irp;
> -        KeMemoryBarrier();
> -
> -        ThreadWake(Pdo->DevicePowerThread);
> -
> -        status = STATUS_PENDING;
> +    //Decide how we're going to handle this IRP.
> +    switch(MinorFunction) {
> +    case IRP_MN_QUERY_POWER:
> +        strategy = XEN_IRP_PASSDOWN_TRANSPARENT; //Dont fail, don't need to check queries.
>           break;
> +    case IRP_MN_SET_POWER:
> +        POWER_STATE_TYPE PowerType = StackLocation->Parameters.Power.Type;
>   
> -    case SystemPowerState:
> -        IoMarkIrpPending(Irp);
> +        switch (PowerType) {
> +        case DevicePowerState:
> +            DEVICE_POWER_STATE DeviceState = StackLocation->Parameters.Power.State.DeviceState;
>   
> -        ASSERT3P(Pdo->SystemPowerIrp, ==, NULL);
> -        Pdo->SystemPowerIrp = Irp;
> -        KeMemoryBarrier();
> -
> -        ThreadWake(Pdo->SystemPowerThread);
> -
> -        status = STATUS_PENDING;
> +            status = IoAcquireRemoveLock(&Pdo->Dx->RemoveLock, Irp);
> +            if (!NT_SUCCESS(status))
> +                goto fail1;
> +
> +            //If fails, then pending removal or already removed,
> +            //freshly initialized (or not). Consider status to return...
> +            HaveRemoveLock = TRUE;
> +
> +            if (DeviceState == __PdoGetDevicePowerState(Pdo)) {
> +                strategy = XEN_IRP_PASSDOWN_TRANSPARENT;
> +            } else if (DeviceState < __PdoGetDevicePowerState(Pdo)) {
> +                strategy = XEN_IRP_HANDLE_INDISPATCH_PASSDOWN;
> +            } else {
> +                strategy = XEN_IRP_HANDLE_INCOMPLETION;
> +            }
> +        case SystemPowerState:
> +            SYSTEM_POWER_STATE SystemState = StackLocation->Parameters.Power.State.SystemState;
> +
> +            status = IoAcquireRemoveLock(&Pdo->Dx->RemoveLock, Irp);
> +            if (!NT_SUCCESS(status))
> +                goto fail1;
> +
> +            //If fails, then pending removal or already removed,
> +            //freshly initialized (or not). Consider status to return...
> +            HaveRemoveLock = TRUE;
> +
> +            if (SystemState == __PdoGetSystemPowerState(Pdo)) {
> +                strategy = XEN_IRP_PASSDOWN_TRANSPARENT;
> +            } else if (SystemState < __PdoGetSystemPowerState(Pdo)) {
> +                strategy = XEN_IRP_HANDLE_INDISPATCH_PASSDOWN;
> +            } else {
> +                strategy = XEN_IRP_HANDLE_INCOMPLETION;
> +            }
>           break;
> -
> +        default:
> +            strategy = XEN_IRP_PASSDOWN_TRANSPARENT;
> +        break;
> +        }
>       default:
> -        IoSkipCurrentIrpStackLocation(Irp);
> -
> -        status = IoCallDriver(Pdo->LowerDeviceObject, Irp);
> -        IoReleaseRemoveLock(&Pdo->Dx->RemoveLock, Irp);
> +        strategy = XEN_IRP_PASSDOWN_TRANSPARENT;
>           break;
>       }
>   
> -    Trace("%s: <==== (%02x:%s) (%08x)\n",
> -          __PdoGetName(Pdo),
> -          MinorFunction,
> -          PowerMinorFunctionName(MinorFunction),
> -          status);
> -
> -done:
> -    return status;
> -
> -fail1:
> -    Error("fail1 (%08x)\n", status);
> +    switch(strategy) {
> +        case XEN_IRP_HANDLE_INDISPATCH_PASSDOWN:
> +            status = PdoDispatchPowerImmediate(Pdo, Irp);
> +            if (!NT_SUCCESS(status))
> +                goto fail2;
> +            //Fallthru.
> +        case XEN_IRP_PASSDOWN_TRANSPARENT:
> +        default:
> +            IoSkipCurrentIrpStackLocation(Irp);
> +            status = IoCallDriver(Pdo->LowerDeviceObject, Irp);
> +            if (!NT_SUCCESS(status))
> +                goto fail3;
>   
> -    Irp->IoStatus.Status = status;
> -    IoCompleteRequest(Irp, IO_NO_INCREMENT);
> +            if (HaveRemoveLock)
> +                IoReleaseRemoveLock(&Pdo->Dx->RemoveLock, Irp);
> +        break;
> +        case XEN_IRP_HANDLE_INCOMPLETION:
> +            IoMarkIrpPending(Irp);
> +            IoCopyCurrentIrpStackLocationToNext(Irp);
> +            IoSetCompletionRoutine(Irp,
> +                                PdoDispatchPowerCompletionV2,
> +                                Pdo,
> +                               TRUE,
> +                               TRUE,
> +                               TRUE);
> +            status = IoCallDriver(Pdo->LowerDeviceObject, Irp);
> +            if (!NT_SUCCESS(status))
> +                goto fail4;
> +            status = STATUS_PENDING; //Must return STATUS_PENDING after IoMarkIrpPending, whatever happens next.
> +        break;
> +    }
>   
>       return status;
> -}
> -
> -static NTSTATUS
> -PdoDispatchDefault(
> -    IN  PXENFILT_PDO    Pdo,
> -    IN  PIRP            Irp
> -    )
> -{
> -    NTSTATUS            status;
>   
> -    status = IoAcquireRemoveLock(&Pdo->Dx->RemoveLock, Irp);
> -    if (!NT_SUCCESS(status))
> -        goto fail1;
> -
> -    IoSkipCurrentIrpStackLocation(Irp);
> -
> -    status = IoCallDriver(Pdo->LowerDeviceObject, Irp);
> -    IoReleaseRemoveLock(&Pdo->Dx->RemoveLock, Irp);
> +fail4:
> +    //Error from lower driver, have remove lock, and completion routine.
> +    //Comp routine shd be called on error to drop remove lock.
> +    Error("fail4 (%08x)\n", status);
> +    status = STATUS_PENDING; //Must return STATUS_PENDING after IoMarkIrpPending, whatever happens next.
> +    return status;
>   
> +fail3:
> +    //Error from lower driver, have remove lock, called down.
> +    Error("fail3 (%08x)\n", status);
> +    if (HaveRemoveLock)
> +        IoReleaseRemoveLock(&Pdo->Dx->RemoveLock, Irp);
>       return status;
>   
> +fail2:
> +    //Immediate error, have remove lock, not called down.
> +    Error("fail2 (%08x)\n", status);
> +    if (HaveRemoveLock)
> +        IoReleaseRemoveLock(&Pdo->Dx->RemoveLock, Irp);
> +    //Fallthru.
> +
>   fail1:
> +    //Immediate error, no remove lock, not called down.
>       Error("fail1 (%08x)\n", status);
>   
>       Irp->IoStatus.Status = status;
> @@ -1907,14 +1551,6 @@ PdoCreate(
>       Pdo->LowerDeviceObject = LowerDeviceObject;
>       Pdo->Type = Type;
>   
> -    status = ThreadCreate(PdoSystemPower, Pdo, &Pdo->SystemPowerThread);
> -    if (!NT_SUCCESS(status))
> -        goto fail4;
> -
> -    status = ThreadCreate(PdoDevicePower, Pdo, &Pdo->DevicePowerThread);
> -    if (!NT_SUCCESS(status))
> -        goto fail5;
> -
>       status = PdoSetDeviceInformation(Pdo);
>       if (!NT_SUCCESS(status))
>           goto fail6;
> @@ -1968,20 +1604,6 @@ fail7:
>   fail6:
>       Error("fail6\n");
>   
> -    ThreadAlert(Pdo->DevicePowerThread);
> -    ThreadJoin(Pdo->DevicePowerThread);
> -    Pdo->DevicePowerThread = NULL;
> -
> -fail5:
> -    Error("fail5\n");
> -
> -    ThreadAlert(Pdo->SystemPowerThread);
> -    ThreadJoin(Pdo->SystemPowerThread);
> -    Pdo->SystemPowerThread = NULL;
> -
> -fail4:
> -    Error("fail4\n");
> -
>       Pdo->Type = XENFILT_EMULATED_OBJECT_TYPE_UNKNOWN;
>       Pdo->PhysicalDeviceObject = NULL;
>       Pdo->LowerDeviceObject = NULL;
> @@ -2040,14 +1662,6 @@ PdoDestroy(
>   
>       PdoClearDeviceInformation(Pdo);
>   
> -    ThreadAlert(Pdo->DevicePowerThread);
> -    ThreadJoin(Pdo->DevicePowerThread);
> -    Pdo->DevicePowerThread = NULL;
> -
> -    ThreadAlert(Pdo->SystemPowerThread);
> -    ThreadJoin(Pdo->SystemPowerThread);
> -    Pdo->SystemPowerThread = NULL;
> -
>       Pdo->Type = XENFILT_EMULATED_OBJECT_TYPE_UNKNOWN;
>       Pdo->PhysicalDeviceObject = NULL;
>       Pdo->LowerDeviceObject = NULL;

   Paul


From win-pv-devel-bounces@lists.xenproject.org Tue Mar 22 14:32:37 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Tue, 22 Mar 2022 14:32:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.293494.498609 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nWfYo-00062z-Lc; Tue, 22 Mar 2022 14:32:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 293494.498609; Tue, 22 Mar 2022 14:32:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nWfYo-00062s-IF; Tue, 22 Mar 2022 14:32:34 +0000
Received: by outflank-mailman (input) for mailman id 293494;
 Tue, 22 Mar 2022 14:32:32 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mH5U=UB=citrix.com=prvs=073f57e0a=martin.harvey@srs-se1.protection.inumbo.net>)
 id 1nWfYm-00062m-Gy
 for win-pv-devel@lists.xenproject.org; Tue, 22 Mar 2022 14:32:32 +0000
Received: from esa3.hc3370-68.iphmx.com (esa3.hc3370-68.iphmx.com
 [216.71.145.155]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e6099d39-a9ec-11ec-a405-831a346695d4;
 Tue, 22 Mar 2022 15:32:31 +0100 (CET)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e6099d39-a9ec-11ec-a405-831a346695d4
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1647959551;
  h=from:to:subject:date:message-id:references:in-reply-to:
   content-transfer-encoding:mime-version;
  bh=O7HVVwU+oR5cUWmtukEWyAt0rOnONIdXL1SWupmyrag=;
  b=hfdUC0jZzNfMXBzPzZ4gIs3iL+Lz0g4CxfyLZAhx8sFyZj6J83hT1+Bm
   ss/on0oGG9tjFO59CshIT9eJ1Sgc5i7FEQYQ+DeFJQONB9LM0K8K564cj
   SPqOfAYLwZd/aC9f6dUK14LphFVQvzqB0ck8RQQyvt9H/zZnuiXE6opu6
   s=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
X-SBRS: 5.1
X-MesageID: 66865701
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:iCxe1KrsgjO1pB8dDF+qlAFVcLZeBmIUZRIvgKrLsJaIsI4StFCzt
 garIBnXb6rbZ2f1KIpyOY3g8B5XucDcx4dlTws4/ithQypE8ZuZCYyVIHmrMnLJJKUvbq7GA
 +byyDXkBJppJpMJjk71atANlVEliefQAOCU5NfsYkidfyc9IMsaoU8lyrZRbrJA24DjWVvW4
 4mq+aUzBXf+s9JKGjNMg068gEsHUMTa4Fv0aXRnOJinFHeH/5UkJMp3yZOZdhMUcaENdgKOf
 M7RzanRw4/s10xF5uVJMFrMWhZirrb6ZWBig5fNMkSoqkAqSicais7XOBeAAKv+Zvrgc91Zk
 b1wWZKMpQgBE4nGhsQMbThkLDhTB69c2ebWHkSfiJnGp6HGWyOEL/RGCUg3OcsT+/ptAHEI/
 vsdQNwPRknd3aTsmuv9E7QywJR4RCXoFNp3VnVIyDjDEfcgB5DKWbvH9PdT3Ssqh9AIFvHbD
 yYcQWQxPEqdPEUWUrsRIIARkuqxlyLESiZntnXFvZdq80bv0AMkhdABN/KKI4fXFK25hH2wu
 WPAunznRx0XKtGb4T6E6W63wP/CmzvhX4AfH6H+8eRl6HWLz3EXIA0bUx28u/bRt6Klc4sBc
 QpOoHNo9PVsshzwJjXgY/GmiFOgtAA8SuRxL9IjxTuP0IuN8jmlJlFRG1atd+canMMxQDUr0
 HqAkNXoGSFjvdWpdJ6NyluHhWjsYHZIdAfucQdBFFJYuIe7/OnfmzqVFr5e/LiJYsoZ8N0a6
 xSDt2AAiroalqbnPI3rrAmc01pASnUkJzPZBzk7vEr4t2uVh6b/PuREDGQ3C94ad+51qXHb4
 BA5dzC2trxmMH10vHXlrB8xNL+o/e2ZFzbXnERiGZIsnxz0pSL8LdAPvW4vfhk0WirhRdMPS
 BWP0e+2zMUOVEZGkIctO97hYyjU5faI+SvZugD8MYMVP8kZmP6v9yByf0+At10BY2B3+ZzTz
 ayzKJ72ZV5DUPwP5GPvG481jO96rghjlDi7bc2qkHyaPU+2OSf9pUEtawDVMIjULcqs/W3oz
 jqoH5DTmksFD7GmPHW/HEx6BQliEEXXzKve8qR/XuWCPhBnCCcmDfrQyqkmYItrg+JekeKgw
 513chYCoLYjrRUr8Tm3V00=
IronPort-HdrOrdr: A9a23:dvaFHaosXlMZIaA/lvBlMTYaV5uQL9V00zEX/kB9WHVpm5Oj+f
 xGzc516farslossSkb6K290dq7MA/hHPlOkMUs1NaZLUTbUQ6TTb2KgrGSugEIdxeOlNK1kJ
 0QCZSWa+eAQmSS7/yKmDVQeuxIqLLsncDY5ts2jU0dNj2CA5sQtjuRYTzrdnGeMTM2Y6bRY6
 DsgfavyQDQHEg/X4CePD0oTuLDr9rEmNbNehgdHSMq7wGIkHeB9KP6OwLw5GZRbxp/hZMZtU
 TVmQ3w4auu99uhzAXH6mPV55NK3PP819p4AtCWgMR9EESstu/oXvUgZ1SxhkF2nAid0idurD
 AKmWZlAy1H0QKTQohym2qr5+Cv6kdp15ao8y7mvZKqm72GeNt9MbsYuWsRSGqp12Mw+N57y6
 5FxGSfqt5eCg7Bhj3045zSWwhtjVfcmwtrrQc/tQ0obWIlUs4ZkWXfxjIhLL4QWCbhrIw3Gu
 hnC8/RoP5QbFOBdnjc+m1i2salUHg/FgqPBhFqgL3e7xFG2HRii0cIzs0WmXkNsJo7Vplf/u
 zBdqBljqtHQMMaZb90QO0BXcy0AGrQRg+kChPYHX33UKUcf37doZ/+57s4oOmsZZwT1ZM33I
 /MVVtJ3FRCDH4Gyff+qKGj3iq9M1lVBw6duv22z6IJyIHBeA==
X-IronPort-AV: E=Sophos;i="5.90,201,1643691600"; 
   d="scan'208";a="66865701"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CQO9n4hqU8ViFFc7xknLj/YN8jt0qcu6rbhWS4fB3MiU68sQGvOZKhKi7nUf42rcn0JHzEKVO4S4iJvLjyLhScK8cGPna7Zy/nQZOVpdy0MjCIYj00PMzypzmXzYl8wNO4e7e793ZTLIlQZ/NxiFZK5KKvduk1mm8JKcC4P2ANoQXaK5xmolJImh6JhY+5JioyvFQdqWz8NxKu/8tp9lBmh/kuzlQaLMIi6QHk1W+iDwAaewRptRP13kuFrc25bDm6QlnzmakTgKFnLdEU3mleoVBqGW2K1ZTLJV3is/2VEZHBNwc6E9Jnww17hGiWa75aNuOjDAKfFb43aIqve0yA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=O7HVVwU+oR5cUWmtukEWyAt0rOnONIdXL1SWupmyrag=;
 b=dLiAr19HsUtofBGFnY8R3nVkZ8SgmPxFN65VuBVUDa1dtlN7mNwyKCeC6QkRrAXRKHfushWsFqohJyi865Jn+hRxTc67mAzC0/5IGB/7k45bgAe1khfJZw2ejCD/DF9FImyNrh5Az6E502ZlG2Gl8ehDIt8bsTj/T051qPf32Ed9DXcOzSgBw1Ya7oZFHNejPgQQqtIsUF9dh9uBGt8EcsbB6EihnsXKk0h2fL2F9RI7rAgZSkvToxXQ+6105sRQT/e6Ax7RgyIIh/Epw19rfAyvaD5SyyBcSWrsBD6dNPiguTMnoElLVGU4u0ghntx9233ocCVcjavBup4/zFLl5w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=O7HVVwU+oR5cUWmtukEWyAt0rOnONIdXL1SWupmyrag=;
 b=nO2udinYMf838eyZu58kCRBUCeVCunSelog2UjjWHFK1bMXgb0Dk2hLpK/acCK1KmHp6s5wx2Kr5x+IE6WZmfOK1e8Rn9uQdIxlYxrlBs3miEZqvqDegdrGV7zIDA3gvVkMyMVVZCA37/1FSIUmHZf4Nt62TaHnhSW40rLe+9bU=
From: Martin Harvey <martin.harvey@citrix.com>
To: "paul@xen.org" <paul@xen.org>, "win-pv-devel@lists.xenproject.org"
	<win-pv-devel@lists.xenproject.org>
Subject: RE: [PATCH] Improve Xenfilt power and default IRP handling.
Thread-Topic: [PATCH] Improve Xenfilt power and default IRP handling.
Thread-Index: AQHYN8VWdc48nzMGh0e89v1joYTp56zLdi4AgAAMNxA=
Date: Tue, 22 Mar 2022 14:32:04 +0000
Message-ID: <MW4PR03MB6347966B82760A88A31765839A179@MW4PR03MB6347.namprd03.prod.outlook.com>
References: <20220314170141.1565-1-Martin.Harvey@citrix.com>
 <da370f45-4ded-6f4e-7c0b-60917d0afa05@gmail.com>
In-Reply-To: <da370f45-4ded-6f4e-7c0b-60917d0afa05@gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: def2cce0-3931-41a3-2ff6-08da0c10bcc3
x-ms-traffictypediagnostic: DM6PR03MB4028:EE_
x-microsoft-antispam-prvs: <DM6PR03MB40289F096D14CFBBBC82FDA09A179@DM6PR03MB4028.namprd03.prod.outlook.com>
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: h0kkJLK/pO7oby28yy6Uy5g5UJomJainiLCBIZuHR4kYYyyELVxz87sXOH6am9vppYS/ZyqiEO/Oicsu6nllNvxkNx9Ip7o9+YQ5AA9W/eQtIO8lKKOozdwi9TNTK+8Crr3aLn7BhxjpJj6tS6y64tqE7MDuY2iq15r83OR939Hmkhe0bYQNXSt5xI1taMsbxicJm54Eu28i7jaMqmr2tHiIPKalVtFckJVOhpEuqf93ER/6S6/XW/z/QSHsJxP61V+mJV5c8jNaiju9TlhAnWVMJDVSGzAFKJ3bjPBcf4G8EL8JG35aiRBGQWIdq3Mijii2G6JPq9Eb38RvNO0GIPMqdUugt5LPITqQaEKjCJOzXC7RYJSS9ho5PBLbm+OOeILV3b4PpEA/hfr/Bi4K2prmuH89tEAH7cta2V4aahoylUxkEGIg6/YVr+71ynr61cTP/jbdXjV3IoHO0jybKPouJWo+GIaYTUR6REwEkEFOwZi3AQFfuF6ylG5RQmNXdTWtae6RGXob+n9Uz5fcfo9zeLavqM9Gd92Swqm574Cn2X7Wwe2yGGAvdLIzmsZs+2PrmgccR5cQT+gk9f+Ig4SxMgJt0oh81wIRD5ITCzHtZE0OZtGxb99+VQphj6RCQIOFwhL1N1vaSkuHZ1GegWb714whqPlitCBVWgeRl+rp2881pjrDHDaEWcB0KfGLokOeXG8fODiRfv8REwjYAg==
x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW4PR03MB6347.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(8936002)(53546011)(186003)(26005)(9686003)(2906002)(6506007)(55016003)(122000001)(38070700005)(33656002)(7696005)(4744005)(83380400001)(82960400001)(71200400001)(76116006)(66946007)(64756008)(8676002)(66446008)(66476007)(110136005)(38100700002)(66556008)(5660300002)(44832011)(52536014)(508600001)(316002)(86362001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?utf-8?B?VCtPUmxOTWNzN3FTenFhdUlWQWdHMmorWWxXQzJOcDVDZlpIUmhTZmxPRXFt?=
 =?utf-8?B?OHdHdVJUNjZNNGxEYlZNanFNekVsUVpwU2gxbjJsdzdMMWhoZXBONm1zVGhs?=
 =?utf-8?B?dnpEREdsSlljUmxWbHlKK0pBbEphVDBQbGI0R0pSQVhGNk11ajg4bS9kNzVl?=
 =?utf-8?B?WXlOZFFJMjgwNU5HYmg4K1FqQW5BZ29sS3NHVVh5QUJLazBhbmh3WVJ1SFl5?=
 =?utf-8?B?UVBGMXVtRkw2a3ZIVTJqSkFGdGUwcGJXV3VGRzMwSm0ydDgwWFpFUFhUcHdM?=
 =?utf-8?B?RlNMWjlZa0thV241bkx1VzVwZUdOV1hsRnJ2aUFyUTlydjZOMFlJRFhNa2F6?=
 =?utf-8?B?NVM3a2Y2M3pjamtBRzd1a011Yms1M082alJPM25jYVpnNGdZMmVrRklZNHVi?=
 =?utf-8?B?NjRiNjl6TFdtWGxXQ2xuWUl4OGlMRTZkK1ROeDY2L0xWbDhjQWh1Y2tWRlhx?=
 =?utf-8?B?ZnlpZWJRWExKUnVxd3BldGo4RTJ4VUtiWE9wdU9xMCtlbndHNTZsMUp0aDFR?=
 =?utf-8?B?YWhNR3FsSTQvK1JEWE82TC9xYUR4dnJVYm4xZWtSM2xJTjE4ME1TUzlNOGlK?=
 =?utf-8?B?NE9ML29pazRFbHhURk53b3RNZVZGL29TakhaUXZRSmhlVWdIUDhjdkFVU3ls?=
 =?utf-8?B?WjllblM3UnN1enJDWGxQaXRFSGhZY1RsNWRRbjRSOXR3U0tjajhhUnFiaE5x?=
 =?utf-8?B?Y09RMDg5bjg5OEV2NVRjU1JjZUdqdGh5anJXd1pMQ2k1dXpZQ1hmS0I0anRM?=
 =?utf-8?B?QXRzbW5OSmtBZzdvcmNCTFJFZzdKNlo2WUpSTVJUcUVFc2Viak1HWUVQY0R1?=
 =?utf-8?B?ZUU1cU1uTHA1YlRMNDJtUGNNYWczWEFTRm9USEdOUUNOOHZyOXlBWlNTWklP?=
 =?utf-8?B?UldBV1dqSHNEb2g5Q21TT2VxWURrblJtdWQyVXdTd0N3RTJXR3J0djZ0MXVJ?=
 =?utf-8?B?dk9oL2t6OWVYOXdSWmorNGhvWml4aW9qN3paeCt5cnplSnNJQ3BoZU43cW1Z?=
 =?utf-8?B?QWFOdFY0NkNpbVJicWNJcmJENFhyTjBLMTQ0UWJDalRXKzJvUTFRU0QxNUVl?=
 =?utf-8?B?SStMR0I1QXJ3cy9qTFNJRWs4L2c5Vm45bmdPbWNWSVJyZUhCR3dMTjhGVmRz?=
 =?utf-8?B?VlQyL3I4VEtOZmRwaWw1NTJDVkFLSTVKc3V6NXJXaDAveVpnemttbW9Eajg0?=
 =?utf-8?B?c3BZNVoyMC9BTndvdmlYQjFKMFlyUVU2cmFnQkNYc2lSSjRybGpRSlVncW15?=
 =?utf-8?B?VFg3MEJVWjVmZmRKNlo5UVc5NE1hY1kvNlduNkhkRmc2eTV0Tk85cDNqWldn?=
 =?utf-8?B?TitQa2pvcUpEOWs4blk5TXFhcU1qczZHeWl4dlpFQ2lYc0dEOHJRWnMraGox?=
 =?utf-8?B?WDV1dHk4REdsNzR0SDlWUThjd0ptWUEwQ2NkZ2t4b0ptZHR3bVJrUW82SUJI?=
 =?utf-8?B?WVdhYlRsL2w3WFJyWllMK2R0M3pBOEZXQjBvR0NaeTY5U25RREpER1dyUGFw?=
 =?utf-8?B?STVxV2p2ckhFcGxBczhsZkZ3RzFNZDNHT1RMSEVJbEpqRUtSc1FPTVlMeHpE?=
 =?utf-8?B?RjRTSTRUTzhGUUE4Tm8ydFEva0puNTRpYU84T0w2bEhIWE9wUFFMb0cyYU4x?=
 =?utf-8?B?MjhMeDd6NnVxaVNwQmRweWdFYlZDREkydWg4NzUreGwzUG5GVGwvVmovY1pW?=
 =?utf-8?B?M1Y5ZXZ1c1BSN1VkVWg5Y3IxaGpXNHM2SHZWUDVWNmcrZVlVK1VuRnlxL1ox?=
 =?utf-8?B?ZitWOU15MG5VUENQbGdRbTZCdG84NHNuN25Ib3BnVjZIWVFVYkYzRjdMNWwv?=
 =?utf-8?B?MENLbjM2b25abExFNE9xRU14UU5PWmVOdzVXazZ1bndTVzFuRnVBUEhCc3Zz?=
 =?utf-8?B?RnNZQXBvOCtRL21ydWJ4cGIvcHM4ajZvMnc1SzZVNHd2YjBkTmU0YWE4T0tP?=
 =?utf-8?B?dTdHN0ZWaWlQUkZvYW1yM0dsTXVoSytOY0prSVh0dFNtUlozaGUrNHBWNm4r?=
 =?utf-8?B?U21OZnJwcU53PT0=?=
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MW4PR03MB6347.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: def2cce0-3931-41a3-2ff6-08da0c10bcc3
X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Mar 2022 14:32:04.9921
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: dZvuFTmxL/dXghZ9BK3IZ9hYvt4ojQIePqS0Zdwgob5w80fdXzS6Fu239AGdAeqJxxFllBZmItmgvAoUoWNnh5jyHqK7GLEXyAd19HiweTg=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB4028
X-OriginatorOrg: citrix.com

DQoNCi0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQpGcm9tOiB3aW4tcHYtZGV2ZWwgPHdpbi1w
di1kZXZlbC1ib3VuY2VzQGxpc3RzLnhlbnByb2plY3Qub3JnPiBPbiBCZWhhbGYgT2YgRHVycmFu
dCwgUGF1bA0KU2VudDogMjIgTWFyY2ggMjAyMiAxMzo0NA0KVG86IHdpbi1wdi1kZXZlbEBsaXN0
cy54ZW5wcm9qZWN0Lm9yZw0KU3ViamVjdDogUmU6IFtQQVRDSF0gSW1wcm92ZSBYZW5maWx0IHBv
d2VyIGFuZCBkZWZhdWx0IElSUCBoYW5kbGluZy4NCg0KDQo+IFRoaXMgaXMgYSB2ZXJ5IGxhcmdl
IHBhdGNoIGdpdmVuIGFic29sdXRlbHkgbm8gZXhwbGFuYXRpb24uIEl0IHJlYWxseSANCj4gbmVl
ZHMgYSAqbXVjaCogbW9yZSBjb21wcmVoZW5zaXZlIGNvbW1pdCBjb21tZW50Li4uIGV2ZW4gYmUg
YnJva2VuIGRvd24gDQo+IGludG8gbXVsdGlwbGUgaW5jcmVtZW50YWwgcGF0Y2hlcywgZS5nLiBt
YXliZSBkb2luZyBGRE8gYW5kIFBETyBzZXBhcmF0ZWx5Lg0KDQpPSy4gVGhhdCdzIGZpbmUuIEkg
bmVlZCB0byB3cml0ZSBhIGxvbmcgZXhwbGFuYXRpb24gYXMgdG8gDQoNCi0gV2h5IHN5bmNocm9u
b3VzbHkgc2VuZGluZyBJUlAncyBkb3duIHdpdGggYSBjb21wbGV0aW9uIHJvdXRpbmUgaXMgY2F1
c2luZyBwcm9ibGVtcy4NCi0gV2h5IHdlIHNob3VsZCBwcm9jZXNzIGFzeW5jaHJvbm91c2x5IGFu
ZCBza2lwIHN0YWNrIGxvY2F0aW9ucy4NCi0gV2hhdCBzZXQgb2YgYnVnY2hlY2tzIHRoaXMgZml4
ZXMuDQoNCi0gSSBhbHNvIG5lZWQgdG8gcmUtd2hxbCBpdC4uLiANCi0gYW5kIE93ZW4gd2FudHMg
dG8gcmVtb3ZlIHRoZSAiU3RyYXRlZ3kiIEVudW0gYW5kIHJlLXdvcmsgdGhlIGNvZGUgc28gaXQn
cyBpbiBhIG1vcmUgY29uc2lzdGVudCBzdHlsZS4uLg0KDQpTbyB3ZSBjYW4gZ2V0IGJhY2sgdG8g
dGhpcyBpbiBhIGJpdCwgYW5kIGl0J2xsIHByb2IgYmUgcmUtd29ya2VkLiBTbGlnaHRseSBrYW1p
a2F6ZSBwYXRjaCBzdWJtaXQsIGp1c3Qgc28geW91IGNvdWxkIGdldCBhIHNuZWFrIHBlZWsuDQoN
Ck1ILg0K


From win-pv-devel-bounces@lists.xenproject.org Tue Mar 22 14:45:47 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Tue, 22 Mar 2022 14:45:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.293508.498624 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nWfla-0007eq-TD; Tue, 22 Mar 2022 14:45:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 293508.498624; Tue, 22 Mar 2022 14:45:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nWfla-0007ej-QF; Tue, 22 Mar 2022 14:45:46 +0000
Received: by outflank-mailman (input) for mailman id 293508;
 Tue, 22 Mar 2022 14:45:45 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UN57=UB=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1nWflY-0007ed-Uf
 for win-pv-devel@lists.xenproject.org; Tue, 22 Mar 2022 14:45:45 +0000
Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com
 [2a00:1450:4864:20::333])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c00a3558-a9ee-11ec-a405-831a346695d4;
 Tue, 22 Mar 2022 15:45:43 +0100 (CET)
Received: by mail-wm1-x333.google.com with SMTP id
 v130-20020a1cac88000000b00389d0a5c511so3027511wme.5
 for <win-pv-devel@lists.xenproject.org>; Tue, 22 Mar 2022 07:45:43 -0700 (PDT)
Received: from [10.139.200.51] ([54.239.6.189])
 by smtp.gmail.com with ESMTPSA id
 g10-20020a5d46ca000000b00203fd86e198sm9396199wrs.96.2022.03.22.07.45.41
 for <win-pv-devel@lists.xenproject.org>
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 22 Mar 2022 07:45:42 -0700 (PDT)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c00a3558-a9ee-11ec-a405-831a346695d4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=message-id:date:mime-version:user-agent:reply-to:subject
         :content-language:to:references:from:in-reply-to
         :content-transfer-encoding;
        bh=f6SXz6lJwb8gFoq+Jl5xcFNLZK0sIXOUgADddVuEoZo=;
        b=a7y3WSB5VkhTruEBeXITWO8JkAbdb9M4R8nOeA+nd37ijzkCQI3Ah8r42oHLL8lNns
         heTGrfMfjfmePoa42NzxC0xRRlsBy7nMC0SAkg8ckN6XTXPX4ypX4wBvPv1RbXJKku6U
         zEaTP4LbGg8oAxGv4pWEaQ/b1yeswO9sx3LEeRiusRmlYwVRmLXtIyvW4fpb++sMJOta
         46G2nsh0vvSw9eVZfTdvRJMHAsN2qk2AuXJQyHlBsdcUlclYVRSwI1OtDQlkcQTrB3cI
         Ib6MoT2zHICyyMBsc+4El3aSWaAuxPJ9MPYqXxo3Fk2foFTpuDo9a78BpzFxTZO5fb7d
         3pBg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:message-id:date:mime-version:user-agent:reply-to
         :subject:content-language:to:references:from:in-reply-to
         :content-transfer-encoding;
        bh=f6SXz6lJwb8gFoq+Jl5xcFNLZK0sIXOUgADddVuEoZo=;
        b=jWG+7Fh6IdobxtlMVtSvVi/ujSJOFcpZUk6nHsWkumx/aPPprHJI0+AnEuQmxdpC8M
         S4oAyhthWHOq51D8bsnjkcdVX5wg2k+OHpnFKeBQ58rvGZE3FabYpxtH/GIQplvvPi5j
         iXGXLqi2PIhdrXaVNofeH2MxmZRsbsttcjCldG1w9++Ie0aCOyRyx7o3QrjwOry9rMgY
         ojEgPNyqCYfXX5uW80Nge02g9Jw/GcawFflIVQKOOhWDbr9UG6I4oQSeuvMoIEuovjRF
         pCk3NCLQ/K5PAfXkvMxhnPsh/ISMAZPjqdhV1S7eYrk0SAHGjaBig6wO1w1OwvyvlVSB
         JHxA==
X-Gm-Message-State: AOAM531He7827PDj3F5wJ7shPubIMoaO0wxOWJkpVWPsUYoxlp3afpEb
	9/5Yr6jC2t/8oIvBQcJ5HrHsPZlrOhLM7w==
X-Google-Smtp-Source: ABdhPJyu6ygUq/De570L/a73PFvYPlN+F2YGIpb7JLsqaosC7mt0xsiIbVNmuwQMw74C6WvSc0do2w==
X-Received: by 2002:a7b:c341:0:b0:37b:ed90:7dad with SMTP id l1-20020a7bc341000000b0037bed907dadmr4100756wmj.138.1647960342620;
        Tue, 22 Mar 2022 07:45:42 -0700 (PDT)
Message-ID: <f52c0683-a870-80b0-7e5b-ec75ef87bded@gmail.com>
Date: Tue, 22 Mar 2022 15:45:41 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.7.0
Reply-To: paul@xen.org
Subject: Re: [PATCH] Windows PV drivers fail to set up RSS when vCPUs > 8
Content-Language: en-US
To: win-pv-devel@lists.xenproject.org
References: <20220322111244.764-1-Martin.Harvey@citrix.com>
From: "Durrant, Paul" <xadimgnik@gmail.com>
In-Reply-To: <20220322111244.764-1-Martin.Harvey@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

I think the key thing that confused me is that you say
'fail to set up RSS when vCPUs > 8' but actually this would be more 
accurately stated as 'fail to set up RSS when vCPU index >= 8'

On 22/03/2022 12:12, Martin Harvey wrote:
> The driver only supports at most 8 queues, however Windows
> can decide to assign vCPU numbers starting from a non-zero
> offset. E.g. vCPU 8,9,10,11 could get assigned to a device
> if you have more than one NIC. The total number of vCPUs
> used by a single device is still less than 8, but the vCPU
> indexes themselves can be greater than 8. The code
> previously incorrectly assumed that individual vCPU
> indexes cannot exceed 8, however a 1:1 mapping between
> vCPU indexes and queues seems to only exist when using
> a single NIC.
> 
> Signed-off-by: Martin Harvey <martin.harvey@citrix.com>
> ---
>   src/xenvif/receiver.c | 168 +++++++++++++++++++++++++++++++++++++-----
>   1 file changed, 148 insertions(+), 20 deletions(-)
> 
> diff --git a/src/xenvif/receiver.c b/src/xenvif/receiver.c
> index 10ac6f5..0c7b32a 100644
> --- a/src/xenvif/receiver.c
> +++ b/src/xenvif/receiver.c
> @@ -106,6 +106,7 @@ typedef struct _XENVIF_RECEIVER_RING {
>       PLIST_ENTRY                 PacketQueue;
>       KDPC                        QueueDpc;
>       ULONG                       QueueDpcs;
> +    PROCESSOR_NUMBER            TargetProcessor;
>       LIST_ENTRY                  PacketComplete;
>       XENVIF_RECEIVER_HASH        Hash;
>   } XENVIF_RECEIVER_RING, *PXENVIF_RECEIVER_RING;
> @@ -2498,6 +2499,9 @@ __ReceiverRingInitialize(
>   
>       KeInitializeThreadedDpc(&(*Ring)->QueueDpc, ReceiverRingQueueDpc, *Ring);
>   
> +    status = KeGetProcessorNumberFromIndex((*Ring)->Index, &(*Ring)->TargetProcessor);
> +    ASSERT(NT_SUCCESS(status));
> +
>       return STATUS_SUCCESS;
>   
>   fail7:
> @@ -2550,6 +2554,45 @@ fail1:
>       return status;
>   }
>   
> +static NTSTATUS
> +__ReceiverRingSetAffinity(

The convention has always been `__` implies `FORCEINLINE`.

> +    IN  PXENVIF_RECEIVER_RING   Ring,
> +    IN  PPROCESSOR_NUMBER       Processor
> +    )
> +{
> +    PXENVIF_RECEIVER            Receiver;
> +    PXENVIF_FRONTEND            Frontend;
> +    NTSTATUS status;
> +
> +    status = STATUS_INVALID_PARAMETER;
> +    if ((Ring == NULL) || (Processor == NULL))
> +        goto fail1;
> +
> +    Receiver = Ring->Receiver;
> +    Frontend = Receiver->Frontend;
> +
> +    /* Always update ring target processor
> +       Actually set affinities if frontend override not present.
> +       Re-bind event-channel if already connected */

Block comment style is messy.

> +
> +    __ReceiverRingAcquireLock(Ring);
> +
> +    Ring->TargetProcessor = *Processor;
> +
> +    /* Don't rebind event channel at this point. */
> +    KeSetTargetProcessorDpcEx(&Ring->PollDpc, &Ring->TargetProcessor);
> +    KeSetTargetProcessorDpcEx(&Ring->QueueDpc, &Ring->TargetProcessor);
> +
> +    __ReceiverRingReleaseLock(Ring);
> +
> +    return STATUS_SUCCESS;
> +
> +fail1:
> +    Error("fail1 (%08x)\n", status);
> +
> +    return status;
> +}
> +
>   static FORCEINLINE NTSTATUS
>   __ReceiverRingConnect(
>       IN  PXENVIF_RECEIVER_RING   Ring
> @@ -2560,7 +2603,6 @@ __ReceiverRingConnect(
>       PFN_NUMBER                  Pfn;
>       CHAR                        Name[MAXNAMELEN];
>       ULONG                       Index;
> -    PROCESSOR_NUMBER            ProcNumber;
>       NTSTATUS                    status;
>   
>       Receiver = Ring->Receiver;
> @@ -2637,16 +2679,17 @@ __ReceiverRingConnect(
>       if (Ring->Channel == NULL)
>           goto fail6;
>   
> -    status = KeGetProcessorNumberFromIndex(Ring->Index, &ProcNumber);
> -    ASSERT(NT_SUCCESS(status));
> +    status = XENBUS_EVTCHN(Bind,
> +                            &Receiver->EvtchnInterface,
> +                            Ring->Channel,
> +                            Ring->TargetProcessor.Group,
> +                            Ring->TargetProcessor.Number);

Looks like indentation is off here.

> +    if (!NT_SUCCESS(status))
> +        Warning("Cound not set initial receiver ring affinity: 0x%x\n", status);

Typo. Plus wouldn't it be useful to have the ring index in the message?

> +    /* You haven't specifically asked for an affinity yet, so just warn. */
>   

Who hasn't asked?

> -    KeSetTargetProcessorDpcEx(&Ring->PollDpc, &ProcNumber);
> -
> -    (VOID) XENBUS_EVTCHN(Bind,
> -                         &Receiver->EvtchnInterface,
> -                         Ring->Channel,
> -                         ProcNumber.Group,
> -                         ProcNumber.Number);
> +    KeSetTargetProcessorDpcEx(&Ring->PollDpc, &Ring->TargetProcessor);
> +    KeSetTargetProcessorDpcEx(&Ring->QueueDpc, &Ring->TargetProcessor);
>   
>       (VOID) XENBUS_EVTCHN(Unmask,
>                            &Receiver->EvtchnInterface,
> @@ -2665,11 +2708,6 @@ __ReceiverRingConnect(
>       if (!NT_SUCCESS(status))
>           goto fail7;
>   
> -    status = KeGetProcessorNumberFromIndex(Ring->Index, &ProcNumber);
> -    ASSERT(NT_SUCCESS(status));
> -
> -    KeSetTargetProcessorDpcEx(&Ring->QueueDpc, &ProcNumber);
> -
>       return STATUS_SUCCESS;
>   
>   fail7:
> @@ -3917,6 +3955,56 @@ fail1:
>       return status;
>   }
>   
> +static NTSTATUS
> +__ReceiverSetQueueAffinities(
> +    IN  PXENVIF_RECEIVER        Receiver,
> +    IN  PPROCESSOR_NUMBER       QueueAffinities,
> +    IN  ULONG                   Count
> +    )
> +{
> +    PXENVIF_FRONTEND        Frontend;
> +    ULONG                   Index;
> +    NTSTATUS                status;
> +    KIRQL                   Irql;
> +
> +    Frontend = Receiver->Frontend;
> +
> +    status = STATUS_INVALID_PARAMETER;
> +
> +    if (QueueAffinities == NULL)
> +        goto fail1;
> +
> +    if (Count > FrontendGetNumQueues(Frontend))
> +        goto fail2;
> +
> +    KeRaiseIrql(DISPATCH_LEVEL, &Irql);
> +
> +    for (Index = 0; Index < Count; Index++) {
> +        PXENVIF_RECEIVER_RING   Ring = Receiver->Ring[Index];
> +
> +        status = __ReceiverRingSetAffinity(Ring, &QueueAffinities[Index]);
> +        if (!NT_SUCCESS(status))
> +            goto fail3;
> +    }
> +
> +    KeLowerIrql(Irql);
> +
> +    return STATUS_SUCCESS;
> +
> +fail3:
> +    KeLowerIrql(Irql);
> +
> +    Error("fail3\n");
> +
> +fail2:
> +    Error("fail2\n");
> +
> +fail1:
> +    Error("fail1 (%08x)\n", status);
> +
> +    return status;
> +}
> +
>   NTSTATUS
>   ReceiverUpdateHashMapping(
>       IN  PXENVIF_RECEIVER    Receiver,
> @@ -3926,10 +4014,15 @@ ReceiverUpdateHashMapping(
>   {
>       PXENVIF_FRONTEND        Frontend;
>       PULONG                  QueueMapping;
> +    PPROCESSOR_NUMBER       QueueAffinities;
>       ULONG                   NumQueues;
> +    ULONG                   QueuesDetermined;
> +    ULONG                   QIndex;
>       ULONG                   Index;
> +    BOOLEAN                 MapEntryDone;
>       NTSTATUS                status;
>   
> +
>       Frontend = Receiver->Frontend;
>   
>       QueueMapping = __ReceiverAllocate(sizeof (ULONG) * Size);
> @@ -3939,26 +4032,61 @@ ReceiverUpdateHashMapping(
>           goto fail1;
>   
>       NumQueues = FrontendGetNumQueues(Frontend);
> +    QueuesDetermined = 0;
> +
> +    QueueAffinities = __ReceiverAllocate(sizeof(PROCESSOR_NUMBER) * NumQueues);
> +    if (QueueAffinities == NULL)
> +        goto fail2;
>   
>       status = STATUS_INVALID_PARAMETER;
> +    /* N^Squared-ish, but performed infrequently */
>       for (Index = 0; Index < Size; Index++) {

The scope of 'MapEntryDone' could be limited to this loop, couldn't it?

> -        QueueMapping[Index] = KeGetProcessorIndexFromNumber(&ProcessorMapping[Index]);
> -
> -        if (QueueMapping[Index] >= NumQueues)
> -            goto fail2;
> +        MapEntryDone = FALSE;
> +        /* Existing queue meets affinity requirement for the mapping at this index? */
> +        for (QIndex = 0; QIndex < QueuesDetermined; QIndex++) {

'QueueIndex' please. It's 'Queue' everywhere else so 'Q' for this looks 
incongruous.

> +            if ((QueueAffinities[QIndex].Group == ProcessorMapping[Index].Group) &&
> +                (QueueAffinities[QIndex].Number == ProcessorMapping[Index].Number)) {
> +                QueueMapping[Index] = QIndex;
> +                MapEntryDone = TRUE;
> +            }
> +        }
> +        if (!MapEntryDone) {
> +            /* New queue "allocation", with new affinity, if possible */
> +            if (QueuesDetermined < NumQueues) {
> +                QIndex = QueuesDetermined;
> +                QueueAffinities[QIndex] = ProcessorMapping[Index];
> +                QueueMapping[Index] = QIndex;
> +                QueuesDetermined ++;

Stray whitespace before the '++'

> +            } else {
> +                goto fail3;

You could do:

if (QueuesDetermined >= NumQueues)
     goto fail3;

QueueIndex = QueuesDetermined;
QueueAffinities[QueueIndex] = ProcessorMapping[Index];
QueueMapping[Index] = QueueIndex;
QueuesDetermined++;

then you would not need the 'else'

   Paul

> +            }
> +        }
>       }
>   
>       status = FrontendSetHashMapping(Frontend, QueueMapping, Size);
>       if (!NT_SUCCESS(status))
> -        goto fail3;
> +        goto fail4;
> +
> +    status = __ReceiverSetQueueAffinities(Receiver, QueueAffinities, QueuesDetermined);
> +    if (!NT_SUCCESS(status))
> +        goto fail5;
>   
>       __ReceiverFree(QueueMapping);
> +    __ReceiverFree(QueueAffinities);
>   
>       return STATUS_SUCCESS;
>   
> +fail5:
> +    Error("fail5\n");
> +
> +fail4:
> +    Error("fail4\n");
> +
>   fail3:
>       Error("fail3\n");
>   
> +    __ReceiverFree(QueueAffinities);
> +
>   fail2:
>       Error("fail2\n");
>   



From win-pv-devel-bounces@lists.xenproject.org Tue Mar 22 14:51:31 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Tue, 22 Mar 2022 14:51:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.293516.498627 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nWfr8-00008q-AT; Tue, 22 Mar 2022 14:51:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 293516.498627; Tue, 22 Mar 2022 14:51:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nWfr8-00008j-7Z; Tue, 22 Mar 2022 14:51:30 +0000
Received: by outflank-mailman (input) for mailman id 293516;
 Tue, 22 Mar 2022 14:51:29 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UN57=UB=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1nWfr7-00008d-A8
 for win-pv-devel@lists.xenproject.org; Tue, 22 Mar 2022 14:51:29 +0000
Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com
 [2a00:1450:4864:20::332])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8db2ad69-a9ef-11ec-a405-831a346695d4;
 Tue, 22 Mar 2022 15:51:28 +0100 (CET)
Received: by mail-wm1-x332.google.com with SMTP id
 bi13-20020a05600c3d8d00b0038c2c33d8f3so3073485wmb.4
 for <win-pv-devel@lists.xenproject.org>; Tue, 22 Mar 2022 07:51:28 -0700 (PDT)
Received: from [10.139.200.51] ([54.239.6.189])
 by smtp.gmail.com with ESMTPSA id
 az26-20020adfe19a000000b00204154a1d1fsm4754237wrb.88.2022.03.22.07.51.27
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 22 Mar 2022 07:51:27 -0700 (PDT)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8db2ad69-a9ef-11ec-a405-831a346695d4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=message-id:date:mime-version:user-agent:reply-to:subject
         :content-language:to:references:from:in-reply-to
         :content-transfer-encoding;
        bh=DY9U9W8X7khAPU0WBkRH/nxyBvUrkRntWztEqbJA83s=;
        b=fbjOoeuAwvh9NhAIDfl4zKbxQW3hANk9N4pMXAH5w6L85K3pEKbqOHl/s4Dd/rDzmp
         NHYFeCHoJpXV+5To54csy71+0cVAeTJExcFtQLVy749/GXNM4BXj+NGBCke/JcnqbVhz
         PjZCMLUmuxhXyTzeyJz5zlBk51sB+6wrf4UeeSBVUNAp0P1RPd/vD0ZL0+9YuYLDt7Fv
         yoio/iG1ZZY+++1GE81cnyb5/iEUm/UESvlmVk8uQ8wZMlZcwWncsyZ9p6gQO+sPgQph
         Sb0gIzBdRlpPPcPT7VE/oIZuPsO0vjI7O/cxEzbqxHMk4tvgTwfw77/JbrL3eBoItgp7
         ozdQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:message-id:date:mime-version:user-agent:reply-to
         :subject:content-language:to:references:from:in-reply-to
         :content-transfer-encoding;
        bh=DY9U9W8X7khAPU0WBkRH/nxyBvUrkRntWztEqbJA83s=;
        b=57T6L7+xkDsdnsqcrxAjc0RW+qEWZ7yaMWXzLSZPeDvzPrY45HWpeE41feqhjC4Z+N
         ee/CchzCj4HoL/rXTUCHuvL/AjzZWmExyMf/cMN1tTSyXsGg0L569N31cWH2hLHdM2hU
         taDeem5JfWLTSDJn8R29tsCXyMXWKBudLhkdsx66Wd8Dhp9sqCntFGkeHupD41IwaqZA
         BZRSgW6LNFovE9YkkAyTcV1btp7VTv9VT9BlK53cyql4nRvnaYiX/QawlmkgO/07gMP/
         CU45sPkTiJpWCamFaltnkw4G74b7Svc3gZoxLD/SRkGNLl+hrta2mFHOE3HbZKORX7vC
         oS9A==
X-Gm-Message-State: AOAM530/9XqPFfZjqmONuz9YiK4NTYxXJJWKLzym9jvNoHp2rX84gUHA
	l5L0IkD/xQb8sIfWrrxyu98=
X-Google-Smtp-Source: ABdhPJxWOZGD2rc1XgzDGBZe4Q27xjiRZw1MveUOvTH7F966RZlkDkmbog/5zq1gEss8duF3NvjDgQ==
X-Received: by 2002:a5d:6d8a:0:b0:204:8aa:309f with SMTP id l10-20020a5d6d8a000000b0020408aa309fmr10552032wrs.38.1647960687721;
        Tue, 22 Mar 2022 07:51:27 -0700 (PDT)
Message-ID: <b2b296b1-ce86-5baf-7935-222b295b35f7@gmail.com>
Date: Tue, 22 Mar 2022 15:51:26 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.7.0
Reply-To: paul@xen.org
Subject: Re: [PATCH] Improve Xenfilt power and default IRP handling.
Content-Language: en-US
To: Martin Harvey <martin.harvey@citrix.com>, "paul@xen.org" <paul@xen.org>,
 "win-pv-devel@lists.xenproject.org" <win-pv-devel@lists.xenproject.org>
References: <20220314170141.1565-1-Martin.Harvey@citrix.com>
 <da370f45-4ded-6f4e-7c0b-60917d0afa05@gmail.com>
 <MW4PR03MB6347966B82760A88A31765839A179@MW4PR03MB6347.namprd03.prod.outlook.com>
From: "Durrant, Paul" <xadimgnik@gmail.com>
In-Reply-To: <MW4PR03MB6347966B82760A88A31765839A179@MW4PR03MB6347.namprd03.prod.outlook.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 22/03/2022 15:32, Martin Harvey wrote:
> 
> 
> -----Original Message-----
> From: win-pv-devel <win-pv-devel-bounces@lists.xenproject.org> On Behalf Of Durrant, Paul
> Sent: 22 March 2022 13:44
> To: win-pv-devel@lists.xenproject.org
> Subject: Re: [PATCH] Improve Xenfilt power and default IRP handling.
> 
> 
>> This is a very large patch given absolutely no explanation. It really
>> needs a *much* more comprehensive commit comment... even be broken down
>> into multiple incremental patches, e.g. maybe doing FDO and PDO separately.
> 
> OK. That's fine. I need to write a long explanation as to
> 
> - Why synchronously sending IRP's down with a completion routine is causing problems.
> - Why we should process asynchronously and skip stack locations.
> - What set of bugchecks this fixes.

Yes, that will make it easier for someone looking back on this to see 
what the fundamental changes were motivated by and how they made things 
better.

> 
> - I also need to re-whql it...
> - and Owen wants to remove the "Strategy" Enum and re-work the code so it's in a more consistent style...
> 

Ok. FWIW I don't think the strategy thing is necessarily a bad idea; I 
think we really just need some better names. If there is another way 
e.g. using function pointers, then that may be neater.

> So we can get back to this in a bit, and it'll prob be re-worked. Slightly kamikaze patch submit, just so you could get a sneak peek.
> 

Well, thanks for the heads up then. Perhaps mark the patch RFC next time 
if you're not really expecting it to be committed? :-)

   Paul




From win-pv-devel-bounces@lists.xenproject.org Wed Mar 23 10:34:21 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Wed, 23 Mar 2022 10:34:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.293838.499316 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nWyJn-0000pt-Or; Wed, 23 Mar 2022 10:34:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 293838.499316; Wed, 23 Mar 2022 10:34:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nWyJn-0000pl-Le; Wed, 23 Mar 2022 10:34:19 +0000
Received: by outflank-mailman (input) for mailman id 293838;
 Wed, 23 Mar 2022 10:34:18 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+0hY=UC=citrix.com=prvs=0740c3c54=martin.harvey@srs-se1.protection.inumbo.net>)
 id 1nWyJm-0000pf-AO
 for win-pv-devel@lists.xenproject.org; Wed, 23 Mar 2022 10:34:18 +0000
Received: from esa3.hc3370-68.iphmx.com (esa3.hc3370-68.iphmx.com
 [216.71.145.155]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b205e960-aa94-11ec-8fbc-03012f2f19d4;
 Wed, 23 Mar 2022 11:33:39 +0100 (CET)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b205e960-aa94-11ec-8fbc-03012f2f19d4
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1648031656;
  h=from:to:cc:subject:date:message-id:mime-version:
   content-transfer-encoding;
  bh=b+NK/nAn6L2LjQHzOCq4S/AIufMclfpDaqjeqo5X8uI=;
  b=GhYnkT0j34wl6lu0nha3K0ywUQqtbh0UPthfJsuiTJSPQmL1Z5NuJ0Z3
   tSZ1UWOYK1H7uuL0PZ4IK6OOKVgylSJ6FcQFPyQ260AYZwxA7QXYEkv25
   2rkeeaw5T/ghKcsT+RYAi0bQdTuvOW32aELglHcj8oHBG9CDvypaHhMDg
   A=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
X-SBRS: 5.1
X-MesageID: 66946669
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:KGXCEaM5hW2/nt7vrR3Dl8FynXyQoLVcMsEvi/4bfWQNrUog1TRTn
 zcYDWCEa6zZZ2OmLdl/boWzpkNX6sWAm4drGQto+SlhQUwRpJueD7x1DKtR0wB+jCHnZBg6h
 ynLQoCYdKjYdleF+lH1dOKJQUBUjclkfJKlYAL/En03FFcMpBsJ00o5wbZi2NQw3bBVPivW0
 T/Mi5yHULOa82Yc3lI8s8pvfzs24ZweEBtB1rAPTagjUG32zhH5P7pGTU2FFFPqQ5E8IwKPb
 72rIIdVXI/u10xF5tuNyt4Xe6CRK1LYFVDmZnF+A8BOjvXez8CbP2lS2Pc0MC9qZzu1c99Z6
 dMSjpexRiAVIrCRgd8jYQhXP3B+IvgTkFPHCSDXXc27ykTHdz3nwul0DVFwNoodkgp1KTgQr
 7pCcmlLN03dwbLtqF64YrAEasALKcDxIIIZ/HFp1y/UHN4tQIzZQrWM7thdtNs1rp4QTKqAO
 ZRIAdZpRD3ubiVpYGY9Mb0dkun5jGHHIjNJmGvA8MLb5ECMlVcsgdABKuH9Zd2MAN1L20qVu
 G/C12D4GQ0BcsySzyKf9XChjfOJmjn0ML/+D5XhqKQs2gfKgDVOVltGDjNXvMVVlGafXY1kI
 E0b3RYysPUrz3KZX9bRDh2n9SvsUgEnZ/JcFOgz6Qeow6XS4hqECmVscgOteODKp+dtG2V0i
 wbhc8fBQGU27ebLES71GqK892vaBMQDEYMVicbopyMh6sKrnow8hwmnoj1LQP/s1Y2d9d0dL
 lm3QMkCa1c70JZjO0aTpwmvb9eQSn7hF19dCuL/BD7N0++BTNT5D7FEEHCChRq6EK6XT0Oao
 F8PkNWE4eYFAPmlzXLRH7RSR+D5uqreb1UwZGKD+bF7qlxBHFb5IOhtDMxWfh81Yq7ohxe3C
 KMshe+hzMAKZyb7BUOGS4mwF94r3cDd+SfNDZjpgi51SsEpLmevpXg2DWbJhjyFuBV8wMkXZ
 MbAGe7xXClyNEiS5GfvLwvr+eRwnX5WKKK6bc2T8ilLJpLFPS/LE+hZaAXSBg37hYvdyDjoH
 x9kH5Pi431ivCfWO0E7LaZ7wYg2EEUG
IronPort-HdrOrdr: A9a23:RJIYha0H0e2F5Wo3G+Si1QqjBLwkLtp133Aq2lEZdPRUGvb4qy
 nIpoV96faUskdpZJhOo6HiBEDtexzhHP1OkO0s1NWZLWvbUQKTRekIh+aP/9SJIVyZygc378
 ddmsZFZuEYdWIK6PrH3A==
X-IronPort-AV: E=Sophos;i="5.90,203,1643691600"; 
   d="scan'208";a="66946669"
From: Martin Harvey <Martin.Harvey@citrix.com>
To: <win-pv-devel@lists.xenproject.org>
CC: Martin Harvey <Martin.Harvey@citrix.com>, Martin Harvey
	<martin.harvey@citrix.com>
Subject: [PATCH] Windows PV drivers fail to set up RSS when vCPU index >= 8
Date: Wed, 23 Mar 2022 10:33:47 +0000
Message-ID: <20220323103347.1891-1-Martin.Harvey@citrix.com>
X-Mailer: git-send-email 2.25.0.windows.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

The driver only supports at most 8 queues, however Windows
can decide to assign vCPU numbers starting from a non-zero
offset. E.g. vCPU 8,9,10,11 could get assigned to a device
if you have more than one NIC. The total number of vCPUs
used by a single device is still less than 8, but the vCPU
indexes themselves can be greater than 8. The code
previously incorrectly assumed that individual vCPU
indexes cannot exceed 8, however a 1:1 mapping between
vCPU indexes and queues seems to only exist when using
a single NIC.

Signed-off-by: Martin Harvey <martin.harvey@citrix.com>
---
 src/xenvif/receiver.c | 161 ++++++++++++++++++++++++++++++++++++------
 1 file changed, 141 insertions(+), 20 deletions(-)

diff --git a/src/xenvif/receiver.c b/src/xenvif/receiver.c
index 10ac6f5..fae9b31 100644
--- a/src/xenvif/receiver.c
+++ b/src/xenvif/receiver.c
@@ -106,6 +106,7 @@ typedef struct _XENVIF_RECEIVER_RING {
     PLIST_ENTRY                 PacketQueue;
     KDPC                        QueueDpc;
     ULONG                       QueueDpcs;
+    PROCESSOR_NUMBER            TargetProcessor;
     LIST_ENTRY                  PacketComplete;
     XENVIF_RECEIVER_HASH        Hash;
 } XENVIF_RECEIVER_RING, *PXENVIF_RECEIVER_RING;
@@ -2498,6 +2499,9 @@ __ReceiverRingInitialize(
 
     KeInitializeThreadedDpc(&(*Ring)->QueueDpc, ReceiverRingQueueDpc, *Ring);
 
+    status = KeGetProcessorNumberFromIndex((*Ring)->Index, &(*Ring)->TargetProcessor);
+    ASSERT(NT_SUCCESS(status));
+
     return STATUS_SUCCESS;
 
 fail7:
@@ -2550,6 +2554,41 @@ fail1:
     return status;
 }
 
+static FORCEINLINE NTSTATUS
+__ReceiverRingSetAffinity(
+    IN  PXENVIF_RECEIVER_RING   Ring,
+    IN  PPROCESSOR_NUMBER       Processor
+    )
+{
+    PXENVIF_RECEIVER            Receiver;
+    PXENVIF_FRONTEND            Frontend;
+    NTSTATUS status;
+
+    status = STATUS_INVALID_PARAMETER;
+    if ((Ring == NULL) || (Processor == NULL))
+        goto fail1;
+
+    Receiver = Ring->Receiver;
+    Frontend = Receiver->Frontend;
+
+    __ReceiverRingAcquireLock(Ring);
+
+    Ring->TargetProcessor = *Processor;
+
+    /* Don't rebind event channel at this point. */
+    KeSetTargetProcessorDpcEx(&Ring->PollDpc, &Ring->TargetProcessor);
+    KeSetTargetProcessorDpcEx(&Ring->QueueDpc, &Ring->TargetProcessor);
+
+    __ReceiverRingReleaseLock(Ring);
+
+    return STATUS_SUCCESS;
+
+fail1:
+    Error("fail1 (%08x)\n", status);
+
+    return status;
+}
+
 static FORCEINLINE NTSTATUS
 __ReceiverRingConnect(
     IN  PXENVIF_RECEIVER_RING   Ring
@@ -2560,7 +2599,6 @@ __ReceiverRingConnect(
     PFN_NUMBER                  Pfn;
     CHAR                        Name[MAXNAMELEN];
     ULONG                       Index;
-    PROCESSOR_NUMBER            ProcNumber;
     NTSTATUS                    status;
 
     Receiver = Ring->Receiver;
@@ -2637,16 +2675,17 @@ __ReceiverRingConnect(
     if (Ring->Channel == NULL)
         goto fail6;
 
-    status = KeGetProcessorNumberFromIndex(Ring->Index, &ProcNumber);
-    ASSERT(NT_SUCCESS(status));
-
-    KeSetTargetProcessorDpcEx(&Ring->PollDpc, &ProcNumber);
+    status = XENBUS_EVTCHN(Bind,
+                           &Receiver->EvtchnInterface,
+                           Ring->Channel,
+                           Ring->TargetProcessor.Group,
+                           Ring->TargetProcessor.Number);
+    if (!NT_SUCCESS(status))
+        Warning("Could not set initial receiver ring affinity: 0x%x for ring %u\n", status, Ring->Index);
+    /* Ring affinity not yet changed from initial CPU number, so just warn. */
 
-    (VOID) XENBUS_EVTCHN(Bind,
-                         &Receiver->EvtchnInterface,
-                         Ring->Channel,
-                         ProcNumber.Group,
-                         ProcNumber.Number);
+    KeSetTargetProcessorDpcEx(&Ring->PollDpc, &Ring->TargetProcessor);
+    KeSetTargetProcessorDpcEx(&Ring->QueueDpc, &Ring->TargetProcessor);
 
     (VOID) XENBUS_EVTCHN(Unmask,
                          &Receiver->EvtchnInterface,
@@ -2665,11 +2704,6 @@ __ReceiverRingConnect(
     if (!NT_SUCCESS(status))
         goto fail7;
 
-    status = KeGetProcessorNumberFromIndex(Ring->Index, &ProcNumber);
-    ASSERT(NT_SUCCESS(status));
-
-    KeSetTargetProcessorDpcEx(&Ring->QueueDpc, &ProcNumber);
-
     return STATUS_SUCCESS;
 
 fail7:
@@ -3917,6 +3951,56 @@ fail1:
     return status;
 }
 
+static NTSTATUS
+__ReceiverSetQueueAffinities(
+    IN  PXENVIF_RECEIVER        Receiver,
+    IN  PPROCESSOR_NUMBER       QueueAffinities,
+    IN  ULONG                   Count
+    )
+{
+    PXENVIF_FRONTEND        Frontend;
+    ULONG                   Index;
+    NTSTATUS                status;
+    KIRQL                   Irql;
+
+    Frontend = Receiver->Frontend;
+
+    status = STATUS_INVALID_PARAMETER;
+
+    if (QueueAffinities == NULL)
+        goto fail1;
+
+    if (Count > FrontendGetNumQueues(Frontend))
+        goto fail2;
+
+    KeRaiseIrql(DISPATCH_LEVEL, &Irql);
+
+    for (Index = 0; Index < Count; Index++) {
+        PXENVIF_RECEIVER_RING   Ring = Receiver->Ring[Index];
+
+        status = __ReceiverRingSetAffinity(Ring, &QueueAffinities[Index]);
+        if (!NT_SUCCESS(status))
+            goto fail3;
+    }
+
+    KeLowerIrql(Irql);
+
+    return STATUS_SUCCESS;
+
+fail3:
+    KeLowerIrql(Irql);
+
+    Error("fail3\n");
+
+fail2:
+    Error("fail2\n");
+
+fail1:
+    Error("fail1 (%08x)\n", status);
+
+    return status;
+}
+
 NTSTATUS
 ReceiverUpdateHashMapping(
     IN  PXENVIF_RECEIVER    Receiver,
@@ -3926,7 +4010,10 @@ ReceiverUpdateHashMapping(
 {
     PXENVIF_FRONTEND        Frontend;
     PULONG                  QueueMapping;
+    PPROCESSOR_NUMBER       QueueAffinities;
     ULONG                   NumQueues;
+    ULONG                   QueuesDetermined;
+    ULONG                   QueueIndex;
     ULONG                   Index;
     NTSTATUS                status;
 
@@ -3939,26 +4026,60 @@ ReceiverUpdateHashMapping(
         goto fail1;
 
     NumQueues = FrontendGetNumQueues(Frontend);
+    QueuesDetermined = 0;
+
+    QueueAffinities = __ReceiverAllocate(sizeof(PROCESSOR_NUMBER) * NumQueues);
+    if (QueueAffinities == NULL)
+        goto fail2;
 
     status = STATUS_INVALID_PARAMETER;
+    /* N^Squared-ish, but performed infrequently */
     for (Index = 0; Index < Size; Index++) {
-        QueueMapping[Index] = KeGetProcessorIndexFromNumber(&ProcessorMapping[Index]);
-
-        if (QueueMapping[Index] >= NumQueues)
-            goto fail2;
+        BOOLEAN MapEntryDone = FALSE;
+        /* Existing queue meets affinity requirement for the mapping at this index? */
+        for (QueueIndex = 0; QueueIndex < QueuesDetermined; QueueIndex++) {
+            if ((QueueAffinities[QueueIndex].Group == ProcessorMapping[Index].Group) &&
+                (QueueAffinities[QueueIndex].Number == ProcessorMapping[Index].Number)) {
+                QueueMapping[Index] = QueueIndex;
+                MapEntryDone = TRUE;
+            }
+        }
+        if (!MapEntryDone) {
+            /* New queue "allocation", with new affinity, if possible */
+            if (QueuesDetermined >= NumQueues)
+                goto fail3;
+
+            QueueIndex = QueuesDetermined;
+            QueueAffinities[QueueIndex] = ProcessorMapping[Index];
+            QueueMapping[Index] = QueueIndex;
+            QueuesDetermined++;
+        }
     }
 
     status = FrontendSetHashMapping(Frontend, QueueMapping, Size);
     if (!NT_SUCCESS(status))
-        goto fail3;
+        goto fail4;
+
+    status = __ReceiverSetQueueAffinities(Receiver, QueueAffinities, QueuesDetermined);
+    if (!NT_SUCCESS(status))
+        goto fail5;
 
     __ReceiverFree(QueueMapping);
+    __ReceiverFree(QueueAffinities);
 
     return STATUS_SUCCESS;
 
+fail5:
+    Error("fail5\n");
+
+fail4:
+    Error("fail4\n");
+
 fail3:
     Error("fail3\n");
 
+    __ReceiverFree(QueueAffinities);
+
 fail2:
     Error("fail2\n");
 
-- 
2.25.0.windows.1



From win-pv-devel-bounces@lists.xenproject.org Wed Mar 23 16:24:44 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Wed, 23 Mar 2022 16:24:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.293954.499573 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nX3mt-0005DE-7T; Wed, 23 Mar 2022 16:24:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 293954.499573; Wed, 23 Mar 2022 16:24:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nX3mt-0005D7-4e; Wed, 23 Mar 2022 16:24:43 +0000
Received: by outflank-mailman (input) for mailman id 293954;
 Wed, 23 Mar 2022 16:24:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=vG3K=UC=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1nX3mr-0005D1-KJ
 for win-pv-devel@lists.xenproject.org; Wed, 23 Mar 2022 16:24:41 +0000
Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com
 [2a00:1450:4864:20::42c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bc3bf177-aac5-11ec-8fbc-03012f2f19d4;
 Wed, 23 Mar 2022 17:24:38 +0100 (CET)
Received: by mail-wr1-x42c.google.com with SMTP id b19so2848796wrh.11
 for <win-pv-devel@lists.xenproject.org>; Wed, 23 Mar 2022 09:24:38 -0700 (PDT)
Received: from [10.139.200.51] ([54.239.6.189])
 by smtp.gmail.com with ESMTPSA id
 i2-20020a05600c354200b0038cb422e3fesm4302643wmq.32.2022.03.23.09.24.37
 for <win-pv-devel@lists.xenproject.org>
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 23 Mar 2022 09:24:37 -0700 (PDT)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bc3bf177-aac5-11ec-8fbc-03012f2f19d4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=message-id:date:mime-version:user-agent:reply-to:subject
         :content-language:to:references:from:in-reply-to
         :content-transfer-encoding;
        bh=f+87F+Ke9yqI9rQZecG0+euqefcYmI0NU0EfusSkqKQ=;
        b=ZXRl7MoRUXLfV1sOZAyZ3f2i7T6Y5IgN/2w3I8+oV0d+iPFFuP6pB+y9p5fE/Q+SkT
         rhT7/vUqJidYNMdltiJwGpv2odn5YY/ZgPY3Kp8V0ynkBzCCAC3Wo33UJWi928gFwbWg
         EoJ8+Wzwxw2wH9Xq89VfrrXF+bKCrPTOu/jtVg6JjZolQFKzXgHpayUMaM2p2vhh3uBi
         F+enFiH0wS8hX9lL7sMZSBixcaHeHNQz+M+Bez0ERdOoDBWiYe9pNVfQgIGaj2oOBrZ/
         V7e9mD/wFhgfYZatXhNjfA7s9PkmdphIEScNfWWbbfRftFHaJgMxa68Xjb8AxT3fCZQP
         XU1w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:message-id:date:mime-version:user-agent:reply-to
         :subject:content-language:to:references:from:in-reply-to
         :content-transfer-encoding;
        bh=f+87F+Ke9yqI9rQZecG0+euqefcYmI0NU0EfusSkqKQ=;
        b=wK5+dAP3khA6pfAvHEtXmMmAPmnVF0APi0m+avOY1tXtpNo+NCMKYHNIcW10q5j8Wv
         rKRvY7pvWG+wwK6e3RT/JD6m0m1cjEmgmii1SAuLSHypehrsTMZyzmSjLYJFrDGNaXlS
         XhhwG4eWJcC5mKHvk4GfqkbXJ113iGvNA4WHIb5bNItdQP4kt568QKHQArH+ZaeOsFUE
         kES3V1m/x43Tmazn0PItJWrY9jcM3nG0AWZlpxVFrrV/OYyKN99C46nsgaLxJlwSgXpF
         msXERmsrl9aQGsI3e6H90Qu7V+yaVH2Wga3ERFu5YEoiuFyltenfy0ZHGlqspUvXn4fm
         ETtA==
X-Gm-Message-State: AOAM530a4diPNpSCAE2URgTM49eqgAD9lQ01QD+2/HqihoQ9fZQsn9up
	fCZB+mNwYkfC9nlsMjh/kOqgSLOI6cfZ2w==
X-Google-Smtp-Source: ABdhPJw82onAxWwd4N9haxiAWW77FgKCRB0vlb0I5VyQDbR/fjpngJCO440c1QxpBp9epqyDvra8gQ==
X-Received: by 2002:a5d:508d:0:b0:203:dac9:d301 with SMTP id a13-20020a5d508d000000b00203dac9d301mr563068wrt.441.1648052678029;
        Wed, 23 Mar 2022 09:24:38 -0700 (PDT)
Message-ID: <5df0c2b5-38ce-7f36-995a-dab9b726b3d7@gmail.com>
Date: Wed, 23 Mar 2022 17:24:36 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.7.0
Reply-To: paul@xen.org
Subject: Re: [PATCH] Windows PV drivers fail to set up RSS when vCPU index >=
 8
Content-Language: en-US
To: win-pv-devel@lists.xenproject.org
References: <20220323103347.1891-1-Martin.Harvey@citrix.com>
From: "Durrant, Paul" <xadimgnik@gmail.com>
In-Reply-To: <20220323103347.1891-1-Martin.Harvey@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 23/03/2022 11:33, Martin Harvey wrote:
> The driver only supports at most 8 queues, however Windows
> can decide to assign vCPU numbers starting from a non-zero
> offset. E.g. vCPU 8,9,10,11 could get assigned to a device
> if you have more than one NIC. The total number of vCPUs
> used by a single device is still less than 8, but the vCPU
> indexes themselves can be greater than 8. The code
> previously incorrectly assumed that individual vCPU
> indexes cannot exceed 8, however a 1:1 mapping between
> vCPU indexes and queues seems to only exist when using
> a single NIC.
> 
> Signed-off-by: Martin Harvey <martin.harvey@citrix.com>

Acked-by: Paul Durrant <paul@xen.org>

... with the following observations...

> ---
>   src/xenvif/receiver.c | 161 ++++++++++++++++++++++++++++++++++++------
>   1 file changed, 141 insertions(+), 20 deletions(-)
> 
> diff --git a/src/xenvif/receiver.c b/src/xenvif/receiver.c
> index 10ac6f5..fae9b31 100644
> --- a/src/xenvif/receiver.c
> +++ b/src/xenvif/receiver.c
> @@ -106,6 +106,7 @@ typedef struct _XENVIF_RECEIVER_RING {
>       PLIST_ENTRY                 PacketQueue;
>       KDPC                        QueueDpc;
>       ULONG                       QueueDpcs;
> +    PROCESSOR_NUMBER            TargetProcessor;
>       LIST_ENTRY                  PacketComplete;
>       XENVIF_RECEIVER_HASH        Hash;
>   } XENVIF_RECEIVER_RING, *PXENVIF_RECEIVER_RING;
> @@ -2498,6 +2499,9 @@ __ReceiverRingInitialize(
>   
>       KeInitializeThreadedDpc(&(*Ring)->QueueDpc, ReceiverRingQueueDpc, *Ring);
>   
> +    status = KeGetProcessorNumberFromIndex((*Ring)->Index, &(*Ring)->TargetProcessor);
> +    ASSERT(NT_SUCCESS(status));
> +
>       return STATUS_SUCCESS;
>   
>   fail7:
> @@ -2550,6 +2554,41 @@ fail1:
>       return status;
>   }
>   
> +static FORCEINLINE NTSTATUS
> +__ReceiverRingSetAffinity(
> +    IN  PXENVIF_RECEIVER_RING   Ring,
> +    IN  PPROCESSOR_NUMBER       Processor
> +    )
> +{
> +    PXENVIF_RECEIVER            Receiver;
> +    PXENVIF_FRONTEND            Frontend;
> +    NTSTATUS status;

misalignment here... I can fix

> +
> +    status = STATUS_INVALID_PARAMETER;
> +    if ((Ring == NULL) || (Processor == NULL))

extra brackets not needed... I will fix

> +        goto fail1;
> +
> +    Receiver = Ring->Receiver;
> +    Frontend = Receiver->Frontend;
> +
> +    __ReceiverRingAcquireLock(Ring);
> +
> +    Ring->TargetProcessor = *Processor;
> +
> +    /* Don't rebind event channel at this point. */
> +    KeSetTargetProcessorDpcEx(&Ring->PollDpc, &Ring->TargetProcessor);
> +    KeSetTargetProcessorDpcEx(&Ring->QueueDpc, &Ring->TargetProcessor);
> +
> +    __ReceiverRingReleaseLock(Ring);
> +
> +    return STATUS_SUCCESS;
> +
> +fail1:
> +    Error("fail1 (%08x)\n", status);
> +
> +    return status;
> +}
> +
>   static FORCEINLINE NTSTATUS
>   __ReceiverRingConnect(
>       IN  PXENVIF_RECEIVER_RING   Ring
> @@ -2560,7 +2599,6 @@ __ReceiverRingConnect(
>       PFN_NUMBER                  Pfn;
>       CHAR                        Name[MAXNAMELEN];
>       ULONG                       Index;
> -    PROCESSOR_NUMBER            ProcNumber;
>       NTSTATUS                    status;
>   
>       Receiver = Ring->Receiver;
> @@ -2637,16 +2675,17 @@ __ReceiverRingConnect(
>       if (Ring->Channel == NULL)
>           goto fail6;
>   
> -    status = KeGetProcessorNumberFromIndex(Ring->Index, &ProcNumber);
> -    ASSERT(NT_SUCCESS(status));
> -
> -    KeSetTargetProcessorDpcEx(&Ring->PollDpc, &ProcNumber);
> +    status = XENBUS_EVTCHN(Bind,
> +                           &Receiver->EvtchnInterface,
> +                           Ring->Channel,
> +                           Ring->TargetProcessor.Group,
> +                           Ring->TargetProcessor.Number);
> +    if (!NT_SUCCESS(status))
> +        Warning("Could not set initial receiver ring affinity: 0x%x for ring %u\n", status, Ring->Index);
> +    /* Ring affinity not yet changed from initial CPU number, so just warn. */

comment is not well placed... I'll fix

>   
> -    (VOID) XENBUS_EVTCHN(Bind,
> -                         &Receiver->EvtchnInterface,
> -                         Ring->Channel,
> -                         ProcNumber.Group,
> -                         ProcNumber.Number);
> +    KeSetTargetProcessorDpcEx(&Ring->PollDpc, &Ring->TargetProcessor);
> +    KeSetTargetProcessorDpcEx(&Ring->QueueDpc, &Ring->TargetProcessor);
>   
>       (VOID) XENBUS_EVTCHN(Unmask,
>                            &Receiver->EvtchnInterface,
> @@ -2665,11 +2704,6 @@ __ReceiverRingConnect(
>       if (!NT_SUCCESS(status))
>           goto fail7;
>   
> -    status = KeGetProcessorNumberFromIndex(Ring->Index, &ProcNumber);
> -    ASSERT(NT_SUCCESS(status));
> -
> -    KeSetTargetProcessorDpcEx(&Ring->QueueDpc, &ProcNumber);
> -
>       return STATUS_SUCCESS;
>   
>   fail7:
> @@ -3917,6 +3951,56 @@ fail1:
>       return status;
>   }
>   
> +static NTSTATUS
> +__ReceiverSetQueueAffinities(

still inconsistent so I'll make this FORCEINLINE

> +    IN  PXENVIF_RECEIVER        Receiver,
> +    IN  PPROCESSOR_NUMBER       QueueAffinities,
> +    IN  ULONG                   Count
> +    )
> +{
> +    PXENVIF_FRONTEND        Frontend;
> +    ULONG                   Index;
> +    NTSTATUS                status;
> +    KIRQL                   Irql;
> +
> +    Frontend = Receiver->Frontend;
> +
> +    status = STATUS_INVALID_PARAMETER;
> +
> +    if (QueueAffinities == NULL)
> +        goto fail1;
> +
> +    if (Count > FrontendGetNumQueues(Frontend))
> +        goto fail2;
> +
> +    KeRaiseIrql(DISPATCH_LEVEL, &Irql);
> +
> +    for (Index = 0; Index < Count; Index++) {
> +        PXENVIF_RECEIVER_RING   Ring = Receiver->Ring[Index];
> +
> +        status = __ReceiverRingSetAffinity(Ring, &QueueAffinities[Index]);
> +        if (!NT_SUCCESS(status))
> +            goto fail3;
> +    }
> +
> +    KeLowerIrql(Irql);
> +
> +    return STATUS_SUCCESS;
> +
> +fail3:
> +    KeLowerIrql(Irql);
> +
> +    Error("fail3\n");

the error message should go immediately after the label... I will fix

> +
> +fail2:
> +    Error("fail2\n");
> +
> +fail1:
> +    Error("fail1 (%08x)\n", status);
> +
> +    return status;
> +}
> +
>   NTSTATUS
>   ReceiverUpdateHashMapping(
>       IN  PXENVIF_RECEIVER    Receiver,
> @@ -3926,7 +4010,10 @@ ReceiverUpdateHashMapping(
>   {
>       PXENVIF_FRONTEND        Frontend;
>       PULONG                  QueueMapping;
> +    PPROCESSOR_NUMBER       QueueAffinities;
>       ULONG                   NumQueues;
> +    ULONG                   QueuesDetermined;
> +    ULONG                   QueueIndex;
>       ULONG                   Index;
>       NTSTATUS                status;
>   
> @@ -3939,26 +4026,60 @@ ReceiverUpdateHashMapping(
>           goto fail1;
>   
>       NumQueues = FrontendGetNumQueues(Frontend);
> +    QueuesDetermined = 0;
> +
> +    QueueAffinities = __ReceiverAllocate(sizeof(PROCESSOR_NUMBER) * NumQueues);
> +    if (QueueAffinities == NULL)
> +        goto fail2;
>   
>       status = STATUS_INVALID_PARAMETER;
> +    /* N^Squared-ish, but performed infrequently */
>       for (Index = 0; Index < Size; Index++) {
> -        QueueMapping[Index] = KeGetProcessorIndexFromNumber(&ProcessorMapping[Index]);
> -
> -        if (QueueMapping[Index] >= NumQueues)
> -            goto fail2;
> +        BOOLEAN MapEntryDone = FALSE;

blank line needed here... I will fix

> +        /* Existing queue meets affinity requirement for the mapping at this index? */

I'll fix this to be a proper sentence

> +        for (QueueIndex = 0; QueueIndex < QueuesDetermined; QueueIndex++) {
> +            if ((QueueAffinities[QueueIndex].Group == ProcessorMapping[Index].Group) &&
> +                (QueueAffinities[QueueIndex].Number == ProcessorMapping[Index].Number)) {
> +                QueueMapping[Index] = QueueIndex;
> +                MapEntryDone = TRUE;
> +            }
> +        }
> +        if (!MapEntryDone) {
> +            /* New queue "allocation", with new affinity, if possible */
> +            if (QueuesDetermined >= NumQueues)
> +                goto fail3;
> +
> +            QueueIndex = QueuesDetermined;
> +            QueueAffinities[QueueIndex] = ProcessorMapping[Index];
> +            QueueMapping[Index] = QueueIndex;
> +            QueuesDetermined++;
> +        }
>       }
>   
>       status = FrontendSetHashMapping(Frontend, QueueMapping, Size);
>       if (!NT_SUCCESS(status))
> -        goto fail3;
> +        goto fail4;
> +
> +    status = __ReceiverSetQueueAffinities(Receiver, QueueAffinities, QueuesDetermined);
> +    if (!NT_SUCCESS(status))
> +        goto fail5;
>   
>       __ReceiverFree(QueueMapping);
> +    __ReceiverFree(QueueAffinities);
>   
>       return STATUS_SUCCESS;
>   
> +fail5:
> +    Error("fail5\n");
> +
> +fail4:
> +    Error("fail4\n");
> +
>   fail3:
>       Error("fail3\n");
>   
> +    __ReceiverFree(QueueAffinities);
> +
>   fail2:
>       Error("fail2\n");
>   



From win-pv-devel-bounces@lists.xenproject.org Wed Mar 23 16:59:18 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Wed, 23 Mar 2022 16:59:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.293972.499610 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nX4KL-0001We-EK; Wed, 23 Mar 2022 16:59:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 293972.499610; Wed, 23 Mar 2022 16:59:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nX4KL-0001WX-BW; Wed, 23 Mar 2022 16:59:17 +0000
Received: by outflank-mailman (input) for mailman id 293972;
 Wed, 23 Mar 2022 16:59:16 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=vG3K=UC=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1nX4KK-0001WR-3u
 for win-pv-devel@lists.xenproject.org; Wed, 23 Mar 2022 16:59:16 +0000
Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com
 [2a00:1450:4864:20::42e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 90d192f0-aaca-11ec-a405-831a346695d4;
 Wed, 23 Mar 2022 17:59:14 +0100 (CET)
Received: by mail-wr1-x42e.google.com with SMTP id w4so2989381wrg.12
 for <win-pv-devel@lists.xenproject.org>; Wed, 23 Mar 2022 09:59:13 -0700 (PDT)
Received: from [10.139.200.51] ([54.239.6.189])
 by smtp.gmail.com with ESMTPSA id
 n65-20020a1c2744000000b003862bfb509bsm4785083wmn.46.2022.03.23.09.59.11
 for <win-pv-devel@lists.xenproject.org>
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 23 Mar 2022 09:59:12 -0700 (PDT)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 90d192f0-aaca-11ec-a405-831a346695d4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=message-id:date:mime-version:user-agent:reply-to:subject
         :content-language:from:to:references:in-reply-to
         :content-transfer-encoding;
        bh=fE2xRxnWCad/eg4OTJrPwfhPE3ysTHEZpfe6TXzWNGA=;
        b=LPeoulZExSw/YTH7R0Mj0R0DclPV7HjzMzmPiRot1L46NaygyzVcAUZvdAn3smrHEf
         +1dg8f5r2QMNO4JzFwJ4lVGM9oG9M4fFGQN7ByDjF/fpe/3G1YT0vW3rdXXZlccr5FDt
         NhLKvMC2IjCllRcep69oYHvdORedT2VuUfBRadCvYVCbUu1DD6c3CvLuKb+5YUTODz2S
         7Utsh70R8rOlF40RfNU3H4BOxABXJ0gRD0RYiNaIkJ9HjQaWGLgunCxQf94yKZ/iyQQH
         Na890DfRMAKoeU4dd8t+W9RhgUUMLKj1vLOD/F9oRHCSW5EAXSeOLdnXUDw5XaBvWnyT
         jTaQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:message-id:date:mime-version:user-agent:reply-to
         :subject:content-language:from:to:references:in-reply-to
         :content-transfer-encoding;
        bh=fE2xRxnWCad/eg4OTJrPwfhPE3ysTHEZpfe6TXzWNGA=;
        b=lK1CzCbcYWtOnKQu73IjDq1PuZMtD4JX4iPRPnk8aLuhp+xmiAzuGee0UMyKCKi2wt
         Jg/XoAO8fSeN0VY/VrpxMxDRH0zuRtbtSsumyd3et5HsOiBF6/nQb5be6y1FvUBZDzQV
         gYl7FRLxNSL6xAAO9eKMsENy6/FDU3ai00Qyo2mPi8jpR1J2o+SRONOdT6nNXgz0p2nN
         Q+PHcrXH3gjsQqHfYcivTglgmH9sDAUE0pJG9hdEJlmWjLVRF6AIimMlkxqo6YSNZI61
         amy6oDF0o9LRbs+vr0tD+OvV0Pfg4PSDBjlxUv6ICElnxNQeQeuv7Jf6qh//j3v3Ew4N
         GmBQ==
X-Gm-Message-State: AOAM531+iWKM+xmLkX9nwuOyF+fkkX6DmOwoMfgUocPtWyPEYC06P+nD
	usKjRiuNoBOc/WVdHXxps14XMBGvPINSZQ==
X-Google-Smtp-Source: ABdhPJwlJ510k3VY6XKbxZy0ogKuvKaNNaQrf2bJr/DhgjYerkcoTrG5HyL6/2AcywZmNqoSc7Ue6A==
X-Received: by 2002:adf:fb47:0:b0:1ed:9f2c:492e with SMTP id c7-20020adffb47000000b001ed9f2c492emr798181wrs.196.1648054753004;
        Wed, 23 Mar 2022 09:59:13 -0700 (PDT)
Message-ID: <96897516-dbe3-9191-ecc9-2f7a47a34e8e@gmail.com>
Date: Wed, 23 Mar 2022 17:59:11 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.7.0
Reply-To: paul@xen.org
Subject: Re: [PATCH] Windows PV drivers fail to set up RSS when vCPU index >=
 8
Content-Language: en-US
From: "Durrant, Paul" <xadimgnik@gmail.com>
To: win-pv-devel@lists.xenproject.org
References: <20220323103347.1891-1-Martin.Harvey@citrix.com>
 <5df0c2b5-38ce-7f36-995a-dab9b726b3d7@gmail.com>
In-Reply-To: <5df0c2b5-38ce-7f36-995a-dab9b726b3d7@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 23/03/2022 17:24, Durrant, Paul wrote:
> On 23/03/2022 11:33, Martin Harvey wrote:
>> The driver only supports at most 8 queues, however Windows
>> can decide to assign vCPU numbers starting from a non-zero
>> offset. E.g. vCPU 8,9,10,11 could get assigned to a device
>> if you have more than one NIC. The total number of vCPUs
>> used by a single device is still less than 8, but the vCPU
>> indexes themselves can be greater than 8. The code
>> previously incorrectly assumed that individual vCPU
>> indexes cannot exceed 8, however a 1:1 mapping between
>> vCPU indexes and queues seems to only exist when using
>> a single NIC.
>>
>> Signed-off-by: Martin Harvey <martin.harvey@citrix.com>
> 
> Acked-by: Paul Durrant <paul@xen.org>
> 
> ... with the following observations...
> 
>> ---
>>   src/xenvif/receiver.c | 161 ++++++++++++++++++++++++++++++++++++------
>>   1 file changed, 141 insertions(+), 20 deletions(-)
>>
>> diff --git a/src/xenvif/receiver.c b/src/xenvif/receiver.c
>> index 10ac6f5..fae9b31 100644
>> --- a/src/xenvif/receiver.c
>> +++ b/src/xenvif/receiver.c
>> @@ -106,6 +106,7 @@ typedef struct _XENVIF_RECEIVER_RING {
>>       PLIST_ENTRY                 PacketQueue;
>>       KDPC                        QueueDpc;
>>       ULONG                       QueueDpcs;
>> +    PROCESSOR_NUMBER            TargetProcessor;
>>       LIST_ENTRY                  PacketComplete;
>>       XENVIF_RECEIVER_HASH        Hash;
>>   } XENVIF_RECEIVER_RING, *PXENVIF_RECEIVER_RING;
>> @@ -2498,6 +2499,9 @@ __ReceiverRingInitialize(
>>       KeInitializeThreadedDpc(&(*Ring)->QueueDpc, 
>> ReceiverRingQueueDpc, *Ring);
>> +    status = KeGetProcessorNumberFromIndex((*Ring)->Index, 
>> &(*Ring)->TargetProcessor);
>> +    ASSERT(NT_SUCCESS(status));
>> +
>>       return STATUS_SUCCESS;
>>   fail7:
>> @@ -2550,6 +2554,41 @@ fail1:
>>       return status;
>>   }
>> +static FORCEINLINE NTSTATUS
>> +__ReceiverRingSetAffinity(
>> +    IN  PXENVIF_RECEIVER_RING   Ring,
>> +    IN  PPROCESSOR_NUMBER       Processor
>> +    )
>> +{
>> +    PXENVIF_RECEIVER            Receiver;
>> +    PXENVIF_FRONTEND            Frontend;
>> +    NTSTATUS status;
> 
> misalignment here... I can fix
> 
>> +
>> +    status = STATUS_INVALID_PARAMETER;
>> +    if ((Ring == NULL) || (Processor == NULL))
> 
> extra brackets not needed... I will fix
> 

Actually this check can go away. This function is static with a single 
caller where both of these arguments can't be NULL.

>> +        goto fail1;
>> +
>> +    Receiver = Ring->Receiver;
>> +    Frontend = Receiver->Frontend;
>> +
>> +    __ReceiverRingAcquireLock(Ring);
>> +
>> +    Ring->TargetProcessor = *Processor;
>> +
>> +    /* Don't rebind event channel at this point. */
>> +    KeSetTargetProcessorDpcEx(&Ring->PollDpc, &Ring->TargetProcessor);
>> +    KeSetTargetProcessorDpcEx(&Ring->QueueDpc, &Ring->TargetProcessor);
>> +
>> +    __ReceiverRingReleaseLock(Ring);
>> +
>> +    return STATUS_SUCCESS;
>> +
>> +fail1:
>> +    Error("fail1 (%08x)\n", status);
>> +
>> +    return status;
>> +}
>> +
>>   static FORCEINLINE NTSTATUS
>>   __ReceiverRingConnect(
>>       IN  PXENVIF_RECEIVER_RING   Ring
>> @@ -2560,7 +2599,6 @@ __ReceiverRingConnect(
>>       PFN_NUMBER                  Pfn;
>>       CHAR                        Name[MAXNAMELEN];
>>       ULONG                       Index;
>> -    PROCESSOR_NUMBER            ProcNumber;
>>       NTSTATUS                    status;
>>       Receiver = Ring->Receiver;
>> @@ -2637,16 +2675,17 @@ __ReceiverRingConnect(
>>       if (Ring->Channel == NULL)
>>           goto fail6;
>> -    status = KeGetProcessorNumberFromIndex(Ring->Index, &ProcNumber);
>> -    ASSERT(NT_SUCCESS(status));
>> -
>> -    KeSetTargetProcessorDpcEx(&Ring->PollDpc, &ProcNumber);
>> +    status = XENBUS_EVTCHN(Bind,
>> +                           &Receiver->EvtchnInterface,
>> +                           Ring->Channel,
>> +                           Ring->TargetProcessor.Group,
>> +                           Ring->TargetProcessor.Number);
>> +    if (!NT_SUCCESS(status))
>> +        Warning("Could not set initial receiver ring affinity: 0x%x 
>> for ring %u\n", status, Ring->Index);
>> +    /* Ring affinity not yet changed from initial CPU number, so just 
>> warn. */
> 
> comment is not well placed... I'll fix
> 
>> -    (VOID) XENBUS_EVTCHN(Bind,
>> -                         &Receiver->EvtchnInterface,
>> -                         Ring->Channel,
>> -                         ProcNumber.Group,
>> -                         ProcNumber.Number);
>> +    KeSetTargetProcessorDpcEx(&Ring->PollDpc, &Ring->TargetProcessor);
>> +    KeSetTargetProcessorDpcEx(&Ring->QueueDpc, &Ring->TargetProcessor);
>>       (VOID) XENBUS_EVTCHN(Unmask,
>>                            &Receiver->EvtchnInterface,
>> @@ -2665,11 +2704,6 @@ __ReceiverRingConnect(
>>       if (!NT_SUCCESS(status))
>>           goto fail7;
>> -    status = KeGetProcessorNumberFromIndex(Ring->Index, &ProcNumber);
>> -    ASSERT(NT_SUCCESS(status));
>> -
>> -    KeSetTargetProcessorDpcEx(&Ring->QueueDpc, &ProcNumber);
>> -
>>       return STATUS_SUCCESS;
>>   fail7:
>> @@ -3917,6 +3951,56 @@ fail1:
>>       return status;
>>   }
>> +static NTSTATUS
>> +__ReceiverSetQueueAffinities(
> 
> still inconsistent so I'll make this FORCEINLINE
> 
>> +    IN  PXENVIF_RECEIVER        Receiver,
>> +    IN  PPROCESSOR_NUMBER       QueueAffinities,
>> +    IN  ULONG                   Count
>> +    )
>> +{
>> +    PXENVIF_FRONTEND        Frontend;
>> +    ULONG                   Index;
>> +    NTSTATUS                status;
>> +    KIRQL                   Irql;
>> +
>> +    Frontend = Receiver->Frontend;
>> +
>> +    status = STATUS_INVALID_PARAMETER;
>> +
>> +    if (QueueAffinities == NULL)
>> +        goto fail1;
>> +
>> +    if (Count > FrontendGetNumQueues(Frontend))
>> +        goto fail2;
>> +
>> +    KeRaiseIrql(DISPATCH_LEVEL, &Irql);
>> +
>> +    for (Index = 0; Index < Count; Index++) {
>> +        PXENVIF_RECEIVER_RING   Ring = Receiver->Ring[Index];
>> +
>> +        status = __ReceiverRingSetAffinity(Ring, 
>> &QueueAffinities[Index]);
>> +        if (!NT_SUCCESS(status))
>> +            goto fail3;
>> +    }
>> +
>> +    KeLowerIrql(Irql);
>> +
>> +    return STATUS_SUCCESS;
>> +
>> +fail3:
>> +    KeLowerIrql(Irql);
>> +
>> +    Error("fail3\n");
> 
> the error message should go immediately after the label... I will fix
> 
>> +
>> +fail2:
>> +    Error("fail2\n");
>> +
>> +fail1:
>> +    Error("fail1 (%08x)\n", status);
>> +
>> +    return status;
>> +}
>> +
>>   NTSTATUS
>>   ReceiverUpdateHashMapping(
>>       IN  PXENVIF_RECEIVER    Receiver,
>> @@ -3926,7 +4010,10 @@ ReceiverUpdateHashMapping(
>>   {
>>       PXENVIF_FRONTEND        Frontend;
>>       PULONG                  QueueMapping;
>> +    PPROCESSOR_NUMBER       QueueAffinities;
>>       ULONG                   NumQueues;
>> +    ULONG                   QueuesDetermined;
>> +    ULONG                   QueueIndex;
>>       ULONG                   Index;
>>       NTSTATUS                status;
>> @@ -3939,26 +4026,60 @@ ReceiverUpdateHashMapping(
>>           goto fail1;
>>       NumQueues = FrontendGetNumQueues(Frontend);
>> +    QueuesDetermined = 0;
>> +
>> +    QueueAffinities = __ReceiverAllocate(sizeof(PROCESSOR_NUMBER) * 
>> NumQueues);
>> +    if (QueueAffinities == NULL)
>> +        goto fail2;
>>       status = STATUS_INVALID_PARAMETER;
>> +    /* N^Squared-ish, but performed infrequently */
>>       for (Index = 0; Index < Size; Index++) {
>> -        QueueMapping[Index] = 
>> KeGetProcessorIndexFromNumber(&ProcessorMapping[Index]);
>> -
>> -        if (QueueMapping[Index] >= NumQueues)
>> -            goto fail2;
>> +        BOOLEAN MapEntryDone = FALSE;
> 
> blank line needed here... I will fix
> 
>> +        /* Existing queue meets affinity requirement for the mapping 
>> at this index? */
> 
> I'll fix this to be a proper sentence
> 
>> +        for (QueueIndex = 0; QueueIndex < QueuesDetermined; 
>> QueueIndex++) {
>> +            if ((QueueAffinities[QueueIndex].Group == 
>> ProcessorMapping[Index].Group) &&
>> +                (QueueAffinities[QueueIndex].Number == 
>> ProcessorMapping[Index].Number)) {
>> +                QueueMapping[Index] = QueueIndex;
>> +                MapEntryDone = TRUE;
>> +            }
>> +        }
>> +        if (!MapEntryDone) {
>> +            /* New queue "allocation", with new affinity, if possible */
>> +            if (QueuesDetermined >= NumQueues)
>> +                goto fail3;
>> +
>> +            QueueIndex = QueuesDetermined;
>> +            QueueAffinities[QueueIndex] = ProcessorMapping[Index];
>> +            QueueMapping[Index] = QueueIndex;
>> +            QueuesDetermined++;
>> +        }
>>       }
>>       status = FrontendSetHashMapping(Frontend, QueueMapping, Size);
>>       if (!NT_SUCCESS(status))
>> -        goto fail3;
>> +        goto fail4;
>> +
>> +    status = __ReceiverSetQueueAffinities(Receiver, QueueAffinities, 
>> QueuesDetermined);
>> +    if (!NT_SUCCESS(status))
>> +        goto fail5;
>>       __ReceiverFree(QueueMapping);
>> +    __ReceiverFree(QueueAffinities);
>>       return STATUS_SUCCESS;
>> +fail5:
>> +    Error("fail5\n");
>> +
>> +fail4:
>> +    Error("fail4\n");
>> +
>>   fail3:
>>       Error("fail3\n");
>> +    __ReceiverFree(QueueAffinities);
>> +
>>   fail2:
>>       Error("fail2\n");
> 



From win-pv-devel-bounces@lists.xenproject.org Thu Mar 24 03:36:00 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Thu, 24 Mar 2022 03:36:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.294130.499957 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nXEGU-0001HL-Sg; Thu, 24 Mar 2022 03:35:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 294130.499957; Thu, 24 Mar 2022 03:35:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nXEGU-0001HD-PI; Thu, 24 Mar 2022 03:35:58 +0000
Received: by outflank-mailman (input) for mailman id 294130;
 Thu, 24 Mar 2022 03:35:57 +0000
Received: from winpvdrvbuild.xenproject.org ([162.242.160.44])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <paul.durrant@xenproject.org>) id 1nXEGT-0001H7-Li
 for win-pv-devel@lists.xenproject.org; Thu, 24 Mar 2022 03:35:57 +0000
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
Date: Thu, 24 Mar 2022 03:35:48 +0000 (UTC)
From: paul.durrant@xenproject.org
To: win-pv-devel@lists.xenproject.org
Message-ID: <19937096.1.1648092951598.JavaMail.WINPVDRVBUILD$@winpvdrvbuild>
Subject: XENVIF-master - Build #193 - Successful
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="----=_Part_0_8599831.1648092950629"
X-Jenkins-Job: XENVIF-master
X-Jenkins-Result: SUCCESS


------=_Part_0_8599831.1648092950629
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

The build system has completed build #193.

Check console output at https://winpvdrvbuild.xenproject.org:8080/job/XENVIF-master/193/console to view the results.
------=_Part_0_8599831.1648092950629--


From win-pv-devel-bounces@lists.xenproject.org Mon Mar 28 10:36:59 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Mon, 28 Mar 2022 10:36:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.295388.502632 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nYmk4-0002Wf-Kr; Mon, 28 Mar 2022 10:36:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 295388.502632; Mon, 28 Mar 2022 10:36:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nYmk4-0002WY-Hl; Mon, 28 Mar 2022 10:36:56 +0000
Received: by outflank-mailman (input) for mailman id 295388;
 Mon, 28 Mar 2022 10:36:55 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=p+xT=UH=citrix.com=prvs=0798cd39f=owen.smith@srs-se1.protection.inumbo.net>)
 id 1nYmk3-0002WS-3Z
 for win-pv-devel@lists.xenproject.org; Mon, 28 Mar 2022 10:36:55 +0000
Received: from esa1.hc3370-68.iphmx.com (esa1.hc3370-68.iphmx.com
 [216.71.145.142]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f9ef8bed-ae82-11ec-8fbc-03012f2f19d4;
 Mon, 28 Mar 2022 12:36:52 +0200 (CEST)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f9ef8bed-ae82-11ec-8fbc-03012f2f19d4
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1648463813;
  h=from:to:cc:subject:date:message-id:mime-version;
  bh=/Qgugioelu0gm22TXQbKepNLBZC8ZroD+crm7kj4Wxg=;
  b=gs31293UzjH3gj7W9W/LxkP24QuceYrYVSYeMeh4G6fe7wXPDnPSFLAu
   dBul5pdke05z+VyNBFmh7jRvAtzbboAoRdzdFrLOwR5mdTiTXcW8JD+89
   nwVFQW3bbsIqWNyDrIxu2Lg8xMIIwe8Gb/AZWMs9Tk9di3Ng6mOTrKRkl
   0=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
X-SBRS: 5.1
X-MesageID: 67719341
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:3gUt8qtY1QeURKpxz8wJrzsstOfnVCVeMUV32f8akzHdYApBsoF/q
 tZmKT3Sbq7bMTGkc4pxPIu09ksDuMLXndQ1HgVu+X08Qn4R+JbJXdiXEBz9bniYRiHhoOOLz
 Cm8hv3odp1coqr0/0/1WlTZhSAgk/nOHNIQMcacUsxLbVYMpBwJ1FQyw4bVvqYy2YLjW1jX6
 YuryyHiEATNNwBcYzp8B52r8HuDjNyq0N/PlgVjDRzjlAa2e0g9VPrzF4noR5fLatA88tqBb
 /TC1NmEElbxpH/BPD8HfoHTKSXmSpaKVeSHZ+E/t6KK2nCurQRquko32WZ1he66RFxlkvgoo
 Oihu6BcRi9xDPHTqaclYiNyNAouHY1JwOaYGT+W5Jn7I03uKxMAwt1rBUAye4YZ5vx2ESdF8
 vlwxDIlN07ZwbjsmfTiF7cq1p9LwMrDZevzvlln0jzfS+0jQIzDa67L+cVZzHE7gcUm8fP2O
 ZVGMGs0ME2ojxtnGFgZD4wRk+aUuFr0U2No8Q+zg7UR/D2GpOB2+Oe0a4eEEjCQfu1Kn0Ddv
 nyD82nnDxUyMN2E1SHD4n+qnvXIny7wRMQVDrLQyxJxqATNnCpJUkRQDAbl56ni4qKjZz5BA
 xQmpjUf7pADzmaMZYndfxKA+SG8+TdJDrK8DNYGwA2Kz6PV5SOQCW4FUiNNZbQaiSMmedA5/
 gTXxo20XFSDpJXQECvArenM8VteLABPdQc/iTk4oRzpCjUJiKU6lVrxQ9lqC8ZZZfWlSGirk
 1hmQMXT7oj/bPLnNY3mpjgrYBr2//AlqzLZAC2NBApJCSsjOOaYi3SAswSz0Bq5BN/xoqO9l
 HYFgdOCy+sFEIuAkieAKM1UQu35u67aameC2gA/d3XEy9hL0yT5FWy3yGsjTHqFz+5eIWO5C
 KMtkV05CGBv0IuCMvYsPtPZ5zUCxqn8D9X1Ps04nfIVCqWdgDSvpXk0DWbJhjiFuBF1zckXZ
 MfKGe7xXC1yIfk2k1KLqxI1jOZDKtYWnjiIG/gWDn2PjNKjWZJiYepcaALVNLFgtstpYmz9q
 r5iCidD8D0GOMXWaSjL648Da1cMKHkwH5ftrMJLMOWEJ2Jb9KsJUpc9HZtJl1RZoplo
IronPort-HdrOrdr: A9a23:fki+4akR5XxYgvdr3eStaEHzDR7pDfIo3DAbv31ZSRFFG/Fw8P
 re+8jztCWE7Ar5PUtKpTnuAsW9qB/nmqKdgrNwAV7BZmfbUQKTRekJgLcKqAeAJwTOssJbyK
 d8Y+xfJbTLfD1HZB/BkWqF+gAbsbu6zJw=
X-IronPort-AV: E=Sophos;i="5.90,217,1643691600"; 
   d="scan'208,217";a="67719341"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=nkqAYR0dobjTupa8EO5dkOUYvt1E4+Su+OoTJ0srrZeisGFMJ5NvosYV8JM35LEPspQs+hraZqpunx9pKhUSEe7vdGsMt7xogoKLBdBkkn/+D2ArQjfRx4LO8P/2pTlhySRJEYg3zZo2ww8ocMMESIgm06DXlMiQZsJ+LaImoha6R29fLPiLbn7D1p2kyynKvUwnfRrTLBu5q2txsL9agDTv4g8Y8m+fokG1xWh12EV/k557qsynGH2/dwtdrPPksOUZrzYlhBoy1WYjDX2bIdJIMVOq2RxQPciIbAJvRIfvGY9uqS+qDmJYn3stemywqcIovgBrwzxwUeiW3pZ6pA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=RXbNQv3JUTGUT71kzAVDl28RtOXM5FUGsuQ1/8KRUEo=;
 b=Ji7UJodHlZoukRRbqD8LtNRk5FMTrdvqdzWGivEs2MYzHoQYhhPUFemsQ6pl/fn1nk2t6k6K5J5R72BdCb4P6dSx+ljRPTzTpmIj3hRzlbeV2dLAu9kldKf02S9V4mmKOVIyAzKr9vdL7Cmm2DQYDOg6Up5M1cjfO3vKYCboEJ6nEVRGVa2zLUgOY3zLhPfWTBZH9Oha0F4b7Vmt2SkiYuYmmPn+Z3QCdV5E00KhlbOkPFq3ywxsq4vLJbtar3WFUsvN4pvI+DopsrEw8tVNpekroRaKbUWaJwJlXiz133pzryd9v+zoVTyx+yi5SX3nx9FVSn8xRzizfS24MRw6XA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=RXbNQv3JUTGUT71kzAVDl28RtOXM5FUGsuQ1/8KRUEo=;
 b=mefKOccXuU52/Sutu+qb0JNseOYnMSqMbQi38oHXZq01yHQLeVmFFuybRZNg2bhPs9X35VTYTGe7u9Cvk555KwDBaJ7Odulw67Dxi4osjuyIXcJcDqWKbedFld1EroXkwtdkevZSaGSHF8TFIlqaxmj4MVgtpwsbhfGoVjduFUE=
From: Owen Smith <owen.smith@citrix.com>
To: "win-pv-devel@lists.xenproject.org" <win-pv-devel@lists.xenproject.org>
CC: Owen Smith <owen.smith@citrix.com>
Subject: TestSign cert passcodes
Thread-Topic: TestSign cert passcodes
Thread-Index: AdhCj3i1MeXHel6mSB2Iu92lqgBbLw==
Date: Mon, 28 Mar 2022 10:36:46 +0000
Message-ID: <DS7PR03MB5477FF7BF632E3D42F10559DFE1D9@DS7PR03MB5477.namprd03.prod.outlook.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 5b08e692-0894-4f6d-929c-08da10a6dbfd
x-ms-traffictypediagnostic: DM6PR03MB4395:EE_
x-microsoft-antispam-prvs: <DM6PR03MB43957361931793CCEE61A53EFE1D9@DM6PR03MB4395.namprd03.prod.outlook.com>
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: ANowrGbEnaYTBqRPyDj1P9U3e9cOPNllOyiFCNxSItaHsZlX9ScfFfjTnPI7gnoL9/9PiD5/E4hEuhS1oXdmOEJQ8vhGTn0TpdD6FbZwP5VxQ9VeVmgPLEhBIaL63movRjQi3ikgJRvT7UQV8eAw3R51tquVdHin4ef4Mz6w8mVBPHdc9w1EjoqYISkXwBbo0aBQarKeedLvTJE11lOLFzda+FLyZYIVUUWZU+HkSpCLtl2H0mCC0OlYuhWCzRBlGXyDaijrgqOVTbM3M/oHCrL9ZxiAYTkYT85nnrh0KhxGmjVSWZAnxyiqHiXOhlu0DzSRhq/w08EzkaIoT+iZDuiQZqNeFuMGk2aDBwLN6pY/eMANZuxVC5WQECCzallMEoJA3Vkz6axA8csPZY2PqXNGq2g+bLFYP4tZGKFv4Aj6TFCdsJE9bMKiefwt88RNiHKu1oGdDZUTPwTCTL5Q7GJmVOe9+BAvfhJ/9t3anqQgE6EuttkdrJFSWzlYofRO8WXqbYc7vumJojMrXthDc5lXtxxm9Zj3VaIZVAh/hsmy6wKOferFK+E9dbZT1GS+LyQS4c2hGor1Fni/AgJwsXafYWLL6YkhwQrHqDjng2E4W2lYpaPdaxCxUJs/94zISA+UnYwAk9qT+hEhnojlwPij4uPBh9k3B67USmIjK/CSmyhJJ0jwV93bsnguyxyENfb1l+S/zWktOLOZSkLJPQ==
x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5477.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(7116003)(71200400001)(7696005)(82960400001)(66556008)(9326002)(3480700007)(66446008)(66476007)(8676002)(76116006)(33656002)(508600001)(52536014)(558084003)(4326008)(86362001)(9686003)(44832011)(6506007)(64756008)(55016003)(2906002)(8936002)(66946007)(316002)(38070700005)(186003)(107886003)(5660300002)(122000001)(38100700002)(26005)(6916009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?iiDfSZYBEiLr63cLr00K4RhdYRuhI+EdRNFwrQ/CTUiH48PGjH+1uovYiCYu?=
 =?us-ascii?Q?Wgf7gds+LdH36VvSrkd7uQIqhQFmhsdKqszv0iNd3tn9Qg77stGNa+C9FbVM?=
 =?us-ascii?Q?5fIK+MD8LjkMluI6NzVPiJWxFDjdeF5iE/sz6PjtbxLxeizW8PrGAvWpKDYC?=
 =?us-ascii?Q?VwiFw85DOpR+Gw6YG7hDSOHRykvyEqhqyaZC/uKrFpl8oJNaRI/d6+4h4UrL?=
 =?us-ascii?Q?WKsjLnpnu5P0WYGhiejfhAW4kUFBQVI3Q/SQml2MbaWS1j0g77J9ti+l9/+r?=
 =?us-ascii?Q?foak+/wuD8fYt3M7xBRfyvfqn9TpsCEr5dtsdS/AJeXu31u1jMJ/ClmxlQRc?=
 =?us-ascii?Q?/okV4W2RZuN9l8h2VavbNm+nSZKxdmZO6TxMXcoboSKR0ldRodNw/BCc/6au?=
 =?us-ascii?Q?LpCbQCs/SwK27u/e14oDxZVhLVzsp9BUWo/UWzKRJgjsQbVN6A00dRQAtuoY?=
 =?us-ascii?Q?K51rh+t9my4nULPSHNyYFq8qAiwVk+CMelfmsVL82fFOxeSWkf+D2+pccO6n?=
 =?us-ascii?Q?lGTTagPCXMw7wrBzHxlwWmxV4tCFOdMOOUxfRHEy0HLd7ImViVOCE0FvZBWH?=
 =?us-ascii?Q?+aq2eVsen8QdkN8m7pBDdfeZIAm89wVCrOBIPJ6IzLDSM/ynEfz/kh/6jkKe?=
 =?us-ascii?Q?22HDwFqB92VVhHdZhc6Tmp1gwaa260n3Bm65HgBBCa4yN33DTpte2DT7alKm?=
 =?us-ascii?Q?YpkKivsclzC6fi+w3r2lAgyQ5MZh/jSdah+N68QKZ5N/Jv3HI++duRVDWavG?=
 =?us-ascii?Q?LUQtb9forQ3D2irvV8vSyGEBDoOQKGJWIeJ5mehDZrfrTz8ak95Uq1lperXx?=
 =?us-ascii?Q?zZYY8pYiZCaTvh1g1G01r22/4R+ved7od2jP6nruum9mXo5ecLLPTw9AXpLD?=
 =?us-ascii?Q?3MdEQkWIMTCYUml52/58lfaSMgbEGY/xkv8D7ain34b+uW08+zkrJjlZgMGA?=
 =?us-ascii?Q?1OQJye0AJTgsokMcur3awL3SEaKnMrLj2itRquXxc80yMkShBOlT278qIiA/?=
 =?us-ascii?Q?p5zgO+xQbl62z2zdcK+t20zdWXjWHk9YN3TlX5mNY93ZaE2D7coYnvlJtQEZ?=
 =?us-ascii?Q?Xg2c1rD2uHl1KFPJhWHa6Y3K5ffEIWSq2PXBJ3H/A7SoqsBZmlEutSb6DeMk?=
 =?us-ascii?Q?QAJwTfkYJYIaPqDXIctovKI+2IvQe1nDvtlLLDW5gRI9pGucCGdMg0fiJm5W?=
 =?us-ascii?Q?7SQNFalOeHSteqDqRGfvZhn7mjA9sOPxAhMNPKC2VtcpvQLKu6hiyUTe5bSf?=
 =?us-ascii?Q?ZmqDYd4FRpqNdt3CbhjnFUU0kSZq+L0c/fpfgt3t2CasSFmffi5ITIgb8FAq?=
 =?us-ascii?Q?Kjjc3WV8U18tLW7KyKySbkEo1cfthanKrcxbehUIAz85RT6hDrPb9a+7h2r2?=
 =?us-ascii?Q?7Flhgw9qWgTlpLy5uB61F9p32uD037BbdiUmZy9MGwYNVICcrsX4UeC+ervb?=
 =?us-ascii?Q?0QaAw5Yk+ikLgpC01dqUPTQe+8055R8G8iHV02W2IfscerKlSA2DgzlX71uH?=
 =?us-ascii?Q?2OhnMSi3uUJF6p3OK4d3A2WAyMlpxhWA0/ZIxCk0qprWGEV0kaINf98Fxl7r?=
 =?us-ascii?Q?aM7jgeAf4B9dq+6E4flPnGVntx79sOO0g2DXCh+vF7NVOKjNfp8nA9befMng?=
 =?us-ascii?Q?9xAgnf1iPYOn1U0ns622BQ1yiJs+3ADPNo4kh5TNSKVjttwi04fU5C0ZkjO7?=
 =?us-ascii?Q?Pr0yeLcmlX3qY4I0YdMRqG4n8bSl35eBkcWN7DE98vBjTnkBkj8/g4hTTSkO?=
 =?us-ascii?Q?hX+Ok6rAmQ=3D=3D?=
Content-Type: multipart/alternative;
	boundary="_000_DS7PR03MB5477FF7BF632E3D42F10559DFE1D9DS7PR03MB5477namp_"
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5477.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 5b08e692-0894-4f6d-929c-08da10a6dbfd
X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Mar 2022 10:36:46.6853
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: ZwdJBl/nzojLUCKC1EfVw5HVTRxwO4pdMygTc6iAy2kyCRm1fyx5meLMr0np3j4OIcYu4aQ9s3kEJ/6PE6coqA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB4395
X-OriginatorOrg: citrix.com

--_000_DS7PR03MB5477FF7BF632E3D42F10559DFE1D9DS7PR03MB5477namp_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Hi,

I've been forced to set up a new dev environment, and am getting an error t=
rying to TestSign the binaries. I'm getting prompted for a passcode for the=
 pfx and don't see this documented in the BUILD.md

Owen

--_000_DS7PR03MB5477FF7BF632E3D42F10559DFE1D9DS7PR03MB5477namp_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr=
osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:=
//www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
<meta name=3D"Generator" content=3D"Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
	{font-family:"Cambria Math";
	panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
	{font-family:Calibri;
	panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0cm;
	font-size:11.0pt;
	font-family:"Calibri",sans-serif;
	mso-fareast-language:EN-US;}
span.EmailStyle17
	{mso-style-type:personal-compose;
	font-family:"Calibri",sans-serif;
	color:windowtext;}
.MsoChpDefault
	{mso-style-type:export-only;
	font-family:"Calibri",sans-serif;
	mso-fareast-language:EN-US;}
@page WordSection1
	{size:612.0pt 792.0pt;
	margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
	{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=3D"EN-GB" link=3D"#0563C1" vlink=3D"#954F72" style=3D"word-wrap:=
break-word">
<div class=3D"WordSection1">
<p class=3D"MsoNormal">Hi,<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">I&#8217;ve been forced to set up a new dev environme=
nt, and am getting an error trying to TestSign the binaries. I&#8217;m gett=
ing prompted for a passcode for the pfx and don&#8217;t see this documented=
 in the BUILD.md<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">Owen<o:p></o:p></p>
</div>
</body>
</html>

--_000_DS7PR03MB5477FF7BF632E3D42F10559DFE1D9DS7PR03MB5477namp_--


From win-pv-devel-bounces@lists.xenproject.org Mon Mar 28 13:22:11 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Mon, 28 Mar 2022 13:22:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.295433.502720 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nYpJw-0005DZ-BZ; Mon, 28 Mar 2022 13:22:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 295433.502720; Mon, 28 Mar 2022 13:22:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nYpJw-0005DS-8p; Mon, 28 Mar 2022 13:22:08 +0000
Received: by outflank-mailman (input) for mailman id 295433;
 Mon, 28 Mar 2022 13:22:06 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=wjt2=UH=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1nYpJu-0005DL-Qw
 for win-pv-devel@lists.xenproject.org; Mon, 28 Mar 2022 13:22:06 +0000
Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com
 [2a00:1450:4864:20::436])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0ee512f2-ae9a-11ec-a405-831a346695d4;
 Mon, 28 Mar 2022 15:22:05 +0200 (CEST)
Received: by mail-wr1-x436.google.com with SMTP id r7so19260983wrc.0
 for <win-pv-devel@lists.xenproject.org>; Mon, 28 Mar 2022 06:22:04 -0700 (PDT)
Received: from ?IPV6:2a00:23c5:5785:9a01:ad9a:ab78:5748:a7ec?
 ([2a00:23c5:5785:9a01:ad9a:ab78:5748:a7ec])
 by smtp.gmail.com with ESMTPSA id
 c12-20020a05600c0a4c00b00381141f4967sm14071746wmq.35.2022.03.28.06.22.02
 for <win-pv-devel@lists.xenproject.org>
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 28 Mar 2022 06:22:03 -0700 (PDT)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0ee512f2-ae9a-11ec-a405-831a346695d4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=message-id:date:mime-version:user-agent:reply-to:subject
         :content-language:to:references:from:in-reply-to
         :content-transfer-encoding;
        bh=QTAcRVTl5uHGlX5gV8SOnrjU04ZwtKFaYsPvWlXSYow=;
        b=aOf61spOYCYktZdFPAmrrmSHhZnP6/svhn8DHWZe9IkADuZu40IM2qjPtn9hbvCw5i
         QrszKhzNMPFqPgy1vho0Kfy7Rqq0kFzKSfIaqKNpa8NwfQ/S3ntUN6vmxzMJqNjHu3Bx
         jCJeCbIpnYNpJ8mZeD1GX1s7Qq/IFR1678ZSdKwlvtiteH7TItMmrpqobs2qVYtiyTh4
         S3a8t70ANua8gD75dRA+50rqPomiz3pfh6iWDbhU50DW3AuxvoeOfA9OssJIYqdoz7Es
         qgguMPexRMBZSBCntZ8/szjMad1nB5sdl/u1hQ5Hv1e1ZOgmXbLzOUy1vcfFiybKgMey
         FSHw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:message-id:date:mime-version:user-agent:reply-to
         :subject:content-language:to:references:from:in-reply-to
         :content-transfer-encoding;
        bh=QTAcRVTl5uHGlX5gV8SOnrjU04ZwtKFaYsPvWlXSYow=;
        b=0WrKspNYGVOkw13jYd9k/IstuAXk2o3/ptzpz08DZpd2YH2uMLOL+ErWFsQ867dNj6
         H2xucreNimCIYzRkkfP5IhRPCEz26IPM5Bf8uudQo3Z/idipdynfUAA96Gv3451xBDBn
         I4dieKUgZfNoy78JMwV5exV2zNHO+55QWi+6gnkKBNJ6YxiR+n7lwo2LrCJGD3zNhnMz
         yviW0IatG13f9KDdfGbEVG/fnrTCgkfn4y25tdhfrUnt9Q1Q1jesbMY/NTEGOi15Bo4f
         jLEIR81ufRKRhZEfgDxy/k08gUCAgkAfJy9ibiNtzg22gVxtBQY+tUwJ1NLYMPg6eCjq
         aA4w==
X-Gm-Message-State: AOAM532ofx5XYXWwRIGhKBHMICK9IwmYVgHzgFrhELEcXglue8aK3nZ+
	KAb29BnIrpO12HSMzy+LjFANBu/PLuI=
X-Google-Smtp-Source: ABdhPJzuVlxPtpaibUL+3g2HbzMQu4JD0b6h7kPubo8WUEABQeAu4wOCW7z7i1gIgDlywPhIbY8udQ==
X-Received: by 2002:a5d:4609:0:b0:203:e792:3add with SMTP id t9-20020a5d4609000000b00203e7923addmr25160053wrq.657.1648473723675;
        Mon, 28 Mar 2022 06:22:03 -0700 (PDT)
Message-ID: <39837734-86b4-e20b-d226-5d8535ceb9ef@gmail.com>
Date: Mon, 28 Mar 2022 14:22:02 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.7.0
Reply-To: paul@xen.org
Subject: Re: TestSign cert passcodes
Content-Language: en-US
To: win-pv-devel@lists.xenproject.org
References: <DS7PR03MB5477FF7BF632E3D42F10559DFE1D9@DS7PR03MB5477.namprd03.prod.outlook.com>
From: "Durrant, Paul" <xadimgnik@gmail.com>
In-Reply-To: <DS7PR03MB5477FF7BF632E3D42F10559DFE1D9@DS7PR03MB5477.namprd03.prod.outlook.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 28/03/2022 11:36, Owen Smith wrote:
> Hi,
> 
> I’ve been forced to set up a new dev environment, and am getting an 
> error trying to TestSign the binaries. I’m getting prompted for a 
> passcode for the pfx and don’t see this documented in the BUILD.md
> 

What happens if you just hit enter? There should be no passcode set.

   Paul



From win-pv-devel-bounces@lists.xenproject.org Mon Mar 28 14:04:41 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Mon, 28 Mar 2022 14:04:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.295457.502786 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nYpz6-0003bZ-R8; Mon, 28 Mar 2022 14:04:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 295457.502786; Mon, 28 Mar 2022 14:04:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nYpz6-0003bS-No; Mon, 28 Mar 2022 14:04:40 +0000
Received: by outflank-mailman (input) for mailman id 295457;
 Mon, 28 Mar 2022 14:04:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=p+xT=UH=citrix.com=prvs=0798cd39f=owen.smith@srs-se1.protection.inumbo.net>)
 id 1nYpz6-0003bK-9W
 for win-pv-devel@lists.xenproject.org; Mon, 28 Mar 2022 14:04:40 +0000
Received: from esa5.hc3370-68.iphmx.com (esa5.hc3370-68.iphmx.com
 [216.71.155.168]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0001d0e4-aea0-11ec-a405-831a346695d4;
 Mon, 28 Mar 2022 16:04:38 +0200 (CEST)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0001d0e4-aea0-11ec-a405-831a346695d4
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1648476277;
  h=from:to:subject:date:message-id:references:in-reply-to:
   content-transfer-encoding:mime-version;
  bh=WtJqGcltbGN64L3L8EH+L0Mqe/1+j8HHct5vzkWXOWA=;
  b=fX5yuxqPvP+JVuCAqMD8kD/oOzxNCTqfDpCzYcj7HiDrrhJ07lz7D7oe
   0J5PzJnyc6+F7FC5M31FiZ2AUo5X0fAcgBMno+yN2p6+/gmqvwIPgg0Tq
   F+oKpC/Vxqr4VGSuvlJUffhcWI/nWaK1IsyvdMUswyBRP7DJswZymHe08
   4=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
X-SBRS: 5.1
X-MesageID: 66780743
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:vSno56n4UPwI7KP76B8L9svo5gyAJkRdPkR7XQ2eYbSJt1+Wr1Gzt
 xIaXjjUbPyONjH0L90ib4238B8H6seAmoBjGlNurC1jESMWpZLJC+rCIxarNUt+DCFioGGLT
 Sk6QoOdRCzhZiaE/n9BCpC48T8kk/vgqoPUUIYoAAgoLeNfYHpn2EoLd9IR2NYy24DiWVPV4
 LsenuWEULOb828sWo4rw/rrRCNH5JwebxtB4zTSzdgS1LPvvyF94KA3fMldHFOhKmVgJcaoR
 v6r8V2M1jixEyHBqD+Suu2TnkUiGtY+NOUV45Zcc/DKbhNq/kTe3kunXRa1hIg+ZzihxrhMJ
 NtxWZOYdwgAIpXHv/8hcT5pEwpfG4pG3LifLi3q2SCT5xWun3rExvxvCAc9PJEC+/YxCmZLn
 RAaAGlTNFbZ3bvwme/lDLk37iggBJCD0Ic3vW5ty3fCCPc6QbjIQrnQ5M8e1zA17ixLNaiAO
 JpBNmY0BPjGS0RGNWgvLcI3pdjyiCTRKxh2r3ONjrVitgA/yyQuieOwYbI5YOeiWsRT216G4
 27L4Wn9KhUbL8CEjyqI9Gq2ge3Clj+9X5gdfJWg/+NuqE2ewCoUEhJ+aLegiaDn0AjkAYsZc
 hFKvHp1xUQvyKC1ZufydgGAq1mfgjw7atp7Mssd6CWKkqWBtm51GVM4ZjJGbdUnsuo/Sjory
 kKFkrvVONB/jFGGYSnDr+nJ9FteLQBQdDZfPnFcEWPp9vG5+OkOYgTzosGP+UJfpvn8AnnOz
 j+Dt0DSbJ1D3JdQh81XEb0q6g9AR6QlrCZovW07vUr/t2uVgbJJgaTyszA3Ct4acO6koqGp5
 iRspiRnxLlm4WuxvCKMWv4RO7qi+uyINjbR6XY2QcVxq2zyoST9Jt4AiN2bGKuPGpxeEdMOS
 BWO0T69GbcJZCf6BUOJS9zZ5zsWIVjISo2+C6G8gitmaZltbg6XlByClmbLt10BZHMEyPllU
 b/CKJ7EJS9DVcxPkWrnL89AgORD7n1vmgvuqWXTkk3PPUy2PyXOF9/o8TKmM4gE0U9ziFiMq
 48EbZrTkX2ykoTWO0HqzGLaFnhTRVATDpHqsc1HMOmFJwttAmY6DPHNh7gmfuRYc259z48kI
 lnVtpdk9WfC
IronPort-HdrOrdr: A9a23:uKf0Da6eMpHGfeZidQPXwdCCI+orL9Y04lQ7vn2ZFiY5TiXIra
 qTdaogviMc0AxhLU3IXOrwcZVpQRvnhONICPoqTMOftWjdySuVxeRZgbcKrAeQahHW2tR07L
 xneaRlEtH8ZGIK1foSjDPIaOrIheP3gpxA5t2ujEuFLzsaG52Ihj0RYmn0Yy4GJng+dOIE+Y
 Knhvav5QDQCkj/Bv7LQ0XtNNKz2+Ej4aiWHyLucCRXlzVm4wnYkoISJyLomis2Yndi0L0m+W
 jAnxfZ4L+/v/yAyhHa33LO9JhN3P/s0MFKCsC0kc4eJi/3hgvAXvUFZ5Sy+BwlquSm6FIni/
 nJugwhONla4W7QeGbdm2qK5yDQlBo+4XrryVuVmlvmuNf4Qy8eA9dMgohIGyGpjHYIjZVRwa
 hP12Kcu4FvFhXQjAH0+tTOXQECrDvRnZNuq59Ps5SRObFuKYN5nMg6xgd4AZ0AFCX15MQOF/
 RvNtjV4LJsfVaTfxnizyVSKIjHZBUONybDZnJHlt2e0jBQknw85VAf3tYjknAJ8494Y4VY5s
 zfW54Yxo1mf4szV+ZQFe0BScy4BijmWhTXKl+fJlzhCeUuJ2/NkZjq+784jdvaK6Dg9KFCza
 gpbWko5FLaOnieV/Fm5ac7rSwllV/NDQgEyallltVEU/PHNfibZRFqjjgV4oydSr4kc7rmss
 2ISdpr6yWKFxq2JW945Xy5Z3AbEwhpbCQ0gKdhZ7vcmLOQFmUd3tarPso6GtLWYHwZspSWOA
 pNYNG0HrQd0qmCYA6yvCTs
X-IronPort-AV: E=Sophos;i="5.90,217,1643691600"; 
   d="scan'208";a="66780743"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=EeN3Jx+dEtTeVkE8+vfqsO/zSawKmNAjMOiwTXdi4gEz312ektgiu775BFdHAVa50TZ/adHnJlFGlLEpFRXgxdsQRdY3OGynwg0RkilNg4ipusgHTVfN3v5rHQ37rY6hAqNlDupyBo9aV8j4nHgRxnF/EHenEX2lTAl8Kfcda3lOzjiZA4F2kUTE+7vT8GAeSJEaPwX59VNtKDtrjrZ+cMzErMWhVob3KzCVlBYhKOmcttpR1mZXTPrWWnjQsbtDlBnRlGh9pSNA45vIcUJCdrPr1/zw4+u2V7+bGdPtI0yfCQDKH7NQYGlgjH0YVt1lkQSyFkYrzBnPb6kSIssbVg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=WtJqGcltbGN64L3L8EH+L0Mqe/1+j8HHct5vzkWXOWA=;
 b=ob1Fd0gU1dYLjVD03YaZBfc2nKeEDWLQ+QfvRzPOS0mR0lXosXVS/OyFZ3yT8/QzFvsdwdKW6kwXBoV2mx+Nokgp6XXTn2CnH3lz6r0L68OBA7tmUaTDycoDy6800pSyCdwwbFciUbrz5ax66xSFT5X2prZcEwrwCwMzv+xUZKsA6o39uT3IZ3OPAQDz8ojeFueXU0L6iImyI1Fa0oAk6Ri55QkBEVfuo7cMWHiQKPaYGwQRQHPPwivKkGK5rqmyDllZv7hfncfOBoA8gUiw40cy2uZkiSgCg3CwwtVE/I0dtj4Z7a+U91JutjlvsrBsfGVDp1FY4fLg2diFLhBxDw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=WtJqGcltbGN64L3L8EH+L0Mqe/1+j8HHct5vzkWXOWA=;
 b=khpynP804ulQ6KlLsOny/5kbpFufjoyRwnAbUYdNy7n3Q0Jgz77J2kjfLQM6K5XnhhilSrjfncZZW3I5LxeXGXhfCHhZk3WKqhqrBZ+GNpfcVVNso7Zi00G6xUG9HaTmFQmFDF2rr0Vsl5bxRrLxt2WIDwu27zBRLo5zoDmsa0k=
From: Owen Smith <owen.smith@citrix.com>
To: "paul@xen.org" <paul@xen.org>, "win-pv-devel@lists.xenproject.org"
	<win-pv-devel@lists.xenproject.org>
Subject: RE: TestSign cert passcodes
Thread-Topic: TestSign cert passcodes
Thread-Index: AdhCj3i1MeXHel6mSB2Iu92lqgBbLwAF1bAAAAEb4vA=
Date: Mon, 28 Mar 2022 14:04:30 +0000
Message-ID: <DS7PR03MB54771A52F9CE1FA229A5605CFE1D9@DS7PR03MB5477.namprd03.prod.outlook.com>
References: <DS7PR03MB5477FF7BF632E3D42F10559DFE1D9@DS7PR03MB5477.namprd03.prod.outlook.com>
 <39837734-86b4-e20b-d226-5d8535ceb9ef@gmail.com>
In-Reply-To: <39837734-86b4-e20b-d226-5d8535ceb9ef@gmail.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 15e3da8c-b4ea-47ae-93d3-08da10c3e12c
x-ms-traffictypediagnostic: CY4PR03MB2757:EE_
x-microsoft-antispam-prvs: <CY4PR03MB2757132C0561976459D6991FFE1D9@CY4PR03MB2757.namprd03.prod.outlook.com>
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: LWeHvxj/933NKjCQw5rCDfuJ5ukfbzRQ+cFRhcQegpSJUpiy1EBtgmIdWhttJnCVpuLzFGbw4E5U6t4+XReKv3DHAEAw5NwWXmweVRk7KzuNAjMgdG/8Apl7/hj12TnIvDfJu7OuHlvAYJ7VGvropx5Vp7xbOEZN5YtRSMRMnvZOVrvcMR7utJ+izc1aDzRE5k2vS2IDj2/Ubkdk56EU4Aa0LYWjY/mO/ro24YflVaEcQKdcPg+eKfr3vhjKbX8jUHEd5Hp+cpkR0tFnaE1U+jEvAxkB9ljT+AMg2RA74xbZ18ycOGlJYlYZ9f+dvoeb8SA0UbvUOGJfkatm7Iq9Lu0+FIZlTkI48J4XeIO1gX9VeIxBZ6YT9+E3zjsW1FO7C4k2yb+ZSCWlh3wKI+F2IOjcsTpkcLmogx4F+2uj42qNX9/pU0vWvwrcQRBDbiazQVuKnCOnqlPMVuDs4X4b8Q3vK6yv6Q9ue4PTFpyE0ThTCLrRsg+drTJxkDxqLPtcS6jdulVqYqfQ3LdMDzwEPIR5g575Pkjlv8aa9f6cImN1kW3aVGtyIMR+0im5mjc5zlAJ3r/zqCiGEAsEjjjuF/DUuF9XT8alkEcrsZKew7uP+6ggUzUIvWEPtYMLfIzC7TL2WvSMb2rkknMufidy1zbXtHXtCr3TH0im24FTh6zUUgwq0E7Hx0QFpoI5iQRrHLBTP+MknMdUaRiIaoXuzA==
x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5477.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(110136005)(38070700005)(83380400001)(4744005)(122000001)(186003)(5660300002)(26005)(316002)(9686003)(38100700002)(6506007)(3480700007)(55016003)(508600001)(33656002)(66446008)(76116006)(8676002)(7116003)(71200400001)(8936002)(2906002)(7696005)(82960400001)(66556008)(53546011)(44832011)(86362001)(66476007)(52536014)(64756008)(66946007);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?utf-8?B?UXdrNFBqVE0vSEtvVVh4ai9mVVNSTkEzVW9VZndKWmtMeHdnTDZPNHBnWUts?=
 =?utf-8?B?VFVneDRJdXVzUjhGdjVYWkRzRW1NaldUTkU0Y01PenhYT000TUMxdWF0emVI?=
 =?utf-8?B?YmdMZ0pxSk55WnNZenFnMlprUGgwTEZ5ZG4wbG5MRmxZVHJJVmlvdGNocUxP?=
 =?utf-8?B?MnJzS0FTT0ltTnlMQ3B3TlByRER5eVBGN1BtdUhJNXdwazFJVEQzMUxSTi9R?=
 =?utf-8?B?Tm5mL0l0MDVLRmNoWW9SWWRvdDNvK0s1ZE81WmRoeThEcXBBM09JMFBEeVR6?=
 =?utf-8?B?RjZWYUhNK0JaelR4YnJRWEpQS1NvNnFpaUZYZWpqRWlIcStiTGs2NllkSVJo?=
 =?utf-8?B?d093U21BRVVSRFBvcHBzekZ3ekJXa0FqRURQeE5zL0kxdVhTUVo2QmlDVk5D?=
 =?utf-8?B?MTdJYVMwN0Q2UEpWMHJwbHZwZkg3NlZmQ1RhN0NxQjRsWFVHWGJTSzF4QlhG?=
 =?utf-8?B?WnNjVVVsVHVueWR4OE1ydzh5bHZKVmJVS2tXYTYyYkVjS1orRjBEd1V6bXRQ?=
 =?utf-8?B?NzRNRlkrYXBScnlaL0E5QlYwY2cyc3FvN3VQZHNZQUk0dUgySjVvdVFDcGJH?=
 =?utf-8?B?UWZSN1pWVnRmR3ZuUWpOU1ltOHBmakVqNjI2SWRBMmRHbFA0a2F5eXdlVXR3?=
 =?utf-8?B?Y2NBTEYrc0c5YzlHOWdFVFVzZlE2YWd1WXJHc0c5S0JLbFdNb0xmSWQxV0k2?=
 =?utf-8?B?ZFVuN1NpY2dEMXhQTVZTazlUSDBHYXNQS0l2cXlIMXZOOUswMTJwUnl2U0lp?=
 =?utf-8?B?MkRGcDhUOG9HRE1lN1d5UjNxYVFQNm91VitBMjRaNy9Nb3lnUVd3MHZ4UVNm?=
 =?utf-8?B?b0VnY3hBOUFEUm5LWTZKK1YzQ0RYdkpjMzQzRHJBa1kxMS9CU2hMQlRoQmp6?=
 =?utf-8?B?a0htN3V2VmI5blhDTzR5djhsOHE4ZVVXMmJ3NVFOdFBzcTV4NE40OHVwQ2Ew?=
 =?utf-8?B?ODVtNzVjbXNaRDFUZENLN1hySHExTUd1QVYxT2VXR2xkSUM1RnQ0azNBYkZK?=
 =?utf-8?B?VTc1aDlFeDZlSDlZOGFuamtEbnF0R3lKaTdGcWJQbDU2Sy9saHZtRURtdFBX?=
 =?utf-8?B?RG0yZ3hGc2F3V01BYmFqTC9pUW9ZWEdKRkM5R1F5SzVLTTlMd1F5YzBGdTZ3?=
 =?utf-8?B?Y2JPV3NlUlNxQ1BoYlYrVm1CSHBlL0VVS1h3dVFQQkhZeWVNVUs2Y2pJejhT?=
 =?utf-8?B?ZEVtNzkvL2FLcUM4ampFUDdMenNHRk5xZFQyQ2svSXp3MFlnUGhWN0NueFV3?=
 =?utf-8?B?L2s3OUtVWDNsanY4K1ZnbUhPSDZJOXVPSm95SGNrRmVwUGZUY2QrNDh5NTBy?=
 =?utf-8?B?cjJuaXp1RHBGVk5ucEsvZ09ERVU2OGJ3MytjSHlEbTVHUmxHaGZTQmg1ZWd1?=
 =?utf-8?B?Y1JTNWt0eG80VWZOQURWRWJHVndDM1J5VWQ5a2d0MDcwYi91ejc4N28rblJy?=
 =?utf-8?B?TUV5NkcwUUF3b2VUdnhFMXNoZ1JremR1cis5eG1nNGFqbmFjU25nZzFjQzhF?=
 =?utf-8?B?bWpiZUdvMDU5MGU0VVREN3BnelBaYnpqejQ3OGlmRUxJdVlLam5wSWJuTWp5?=
 =?utf-8?B?ZTNnQ3lndjM1ZmlYWEtUa3NlcEMwdjNyNUs1QXpSTUxVRmIveFNzOERxSUt2?=
 =?utf-8?B?RWhXb0xFYlhjWEhmcG9URHgwODJrY2JQMVYrcjlBeDYxanVKS1dtT21JclJB?=
 =?utf-8?B?RW8weHlxQ0ZLRWpHZzlteWtOSzhXcEVXZU02emRxY3QxQ0VTMnlLbWxLNWFT?=
 =?utf-8?B?RjBLMFZRQTlsMWJhQXlUb25wZXBwcUN4RkV3VEppN3ViOGlXcGRheExEd2dm?=
 =?utf-8?B?TW9rV2RLZ3h3Q1VPeFNPaWJGRHZ5RHY4YnZLeGhwdVFOeUhrbjUrbllHNnly?=
 =?utf-8?B?RENidGlUVWhqRFQzRS9DZWZUMG0zdytxVHRHYzdWNUhLOGsxc1ZPT2VtcGtL?=
 =?utf-8?B?MXJUdys5RjQwZXJBV0poVTVVYzBlOXRTWi9hck1yTUVJZmhsYTVqangvWDZM?=
 =?utf-8?B?Nlkvb2laeVB2QStzaXVjbjRKUmo0MWZYVDNCc0pDMVVxanduVm1VWGhxSDJM?=
 =?utf-8?B?YjloQzJ0Y2FCTm9HSmp0dHM5OUNLZlNGYVo0QTFFenc0bGpWZGdnRk9NQW4v?=
 =?utf-8?B?THdTeU43UlZGb0YwRWRZYkFjOHBqMlA0SkFjVVhsL3dvWU9YVHgyRkgySjZ3?=
 =?utf-8?B?b3dROWpVemQwbU40UlFQUnRSbi82S1Q1R2RkTnBSWmF4RHQ0N0pkbUZvTzNC?=
 =?utf-8?B?VDN5SXFVaEp3alNFdU9RZjlMOE1UZnJHTGo5T3kvbEZ1Si9DOXlYOXlqaWdl?=
 =?utf-8?B?eU1tZkxvdVNzRHh6RWUwUy9jaXJpZHRkekNHTDJXVXUwLytBeFBpdz09?=
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5477.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 15e3da8c-b4ea-47ae-93d3-08da10c3e12c
X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Mar 2022 14:04:30.7856
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 8OfrHTFd4IemmL2y2+t/Sf0537fayaPCKkhrOqXEPeSDwCU8byiwPSX4km8Im2DzRL4yMBEQELO7wdZIjRRu0w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR03MB2757
X-OriginatorOrg: citrix.com

LS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCkZyb206IHdpbi1wdi1kZXZlbCA8d2luLXB2LWRl
dmVsLWJvdW5jZXNAbGlzdHMueGVucHJvamVjdC5vcmc+IE9uIEJlaGFsZiBPZiBEdXJyYW50LCBQ
YXVsDQpTZW50OiAyOCBNYXJjaCAyMDIyIDE0OjIyDQpUbzogd2luLXB2LWRldmVsQGxpc3RzLnhl
bnByb2plY3Qub3JnDQpTdWJqZWN0OiBSZTogVGVzdFNpZ24gY2VydCBwYXNzY29kZXMNCg0KW0NB
VVRJT04gLSBFWFRFUk5BTCBFTUFJTF0gRE8gTk9UIHJlcGx5LCBjbGljayBsaW5rcywgb3Igb3Bl
biBhdHRhY2htZW50cyB1bmxlc3MgeW91IGhhdmUgdmVyaWZpZWQgdGhlIHNlbmRlciBhbmQga25v
dyB0aGUgY29udGVudCBpcyBzYWZlLg0KDQpPbiAyOC8wMy8yMDIyIDExOjM2LCBPd2VuIFNtaXRo
IHdyb3RlOg0KPiBIaSwNCj4gDQo+IEnigJl2ZSBiZWVuIGZvcmNlZCB0byBzZXQgdXAgYSBuZXcg
ZGV2IGVudmlyb25tZW50LCBhbmQgYW0gZ2V0dGluZyBhbiANCj4gZXJyb3IgdHJ5aW5nIHRvIFRl
c3RTaWduIHRoZSBiaW5hcmllcy4gSeKAmW0gZ2V0dGluZyBwcm9tcHRlZCBmb3IgYSANCj4gcGFz
c2NvZGUgZm9yIHRoZSBwZnggYW5kIGRvbuKAmXQgc2VlIHRoaXMgZG9jdW1lbnRlZCBpbiB0aGUg
QlVJTEQubWQNCj4gDQoNCldoYXQgaGFwcGVucyBpZiB5b3UganVzdCBoaXQgZW50ZXI/IFRoZXJl
IHNob3VsZCBiZSBubyBwYXNzY29kZSBzZXQuDQoiZXJyb3I6IEludmFsaWQgY2VydGlmaWNhdGUi
DQoNCklmIEkgcmVnZW5lcmF0ZSB0aGUgcGZ4LCBpdCB3b3JrcyBhbmQgdGVzdCBzaWducyB0aGUg
YmluYXJpZXMNCg0KQ29tcGFyaW5nIHRoZSByZWdlbmVyYXRlZCBwZnggd2l0aCB0aGUgb3JpZ2lu
YWwsIEkgY2FuIHNlZSBubyBpbW1lZGlhdGUgZGlmZmVyZW5jZXMgKG90aGVyIHRoYW4gc3RhcnQg
ZGF0ZSBhbmQgc2VyaWFsIG51bWJlcikNCg0KT3dlbg0KDQoNCg0KSWYgSSBkb27igJl0IGVudGVy
IGEgcGFzc2NvZGUgb3IgZW50ZXIgYW55dGhpbmcsIEkgZ2V0IGEgc2lnbmluZyBmYWlsdXJlDQoN
Cg0KICAgUGF1bA0KDQoNCg==


From win-pv-devel-bounces@lists.xenproject.org Mon Mar 28 14:42:55 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Mon, 28 Mar 2022 14:42:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.295477.502818 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nYqa7-0000CX-A7; Mon, 28 Mar 2022 14:42:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 295477.502818; Mon, 28 Mar 2022 14:42:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nYqa7-0000CP-6y; Mon, 28 Mar 2022 14:42:55 +0000
Received: by outflank-mailman (input) for mailman id 295477;
 Mon, 28 Mar 2022 14:42:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=wjt2=UH=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1nYqa5-0000CF-Pa
 for win-pv-devel@lists.xenproject.org; Mon, 28 Mar 2022 14:42:53 +0000
Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com
 [2a00:1450:4864:20::335])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 58a7d684-aea5-11ec-8fbc-03012f2f19d4;
 Mon, 28 Mar 2022 16:42:52 +0200 (CEST)
Received: by mail-wm1-x335.google.com with SMTP id q20so8545581wmq.1
 for <win-pv-devel@lists.xenproject.org>; Mon, 28 Mar 2022 07:42:52 -0700 (PDT)
Received: from ?IPV6:2a00:23c5:5785:9a01:ad9a:ab78:5748:a7ec?
 ([2a00:23c5:5785:9a01:ad9a:ab78:5748:a7ec])
 by smtp.gmail.com with ESMTPSA id
 l12-20020a056000022c00b00203ee262d12sm11995183wrz.116.2022.03.28.07.42.51
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 28 Mar 2022 07:42:51 -0700 (PDT)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 58a7d684-aea5-11ec-8fbc-03012f2f19d4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=message-id:date:mime-version:user-agent:reply-to:subject
         :content-language:to:references:from:in-reply-to
         :content-transfer-encoding;
        bh=XX5rHighIskvO6S2Qrw71pxmwvjWU7Eftr8yjX/vn+s=;
        b=FO6SKKzKOmxUbvs5TLrKGTfolBpcNFyhGsqQpJZjsvpAt/4e+n8X0t1RfNzPkY/q4A
         JRugQ5uD+DiNB4eFb1+JA+RSKKwxghlRtZ4Wa7hktGy70a6QNXqVQ0NT+RkFPggenfIt
         a7U+5ra1X27tPU6N8Tx1lu55T74wkPoGnfoSveVfnS3raUc7a9uLWD0gEZ04Ha1R81sj
         TXkoAkP+KuCDB0CStqlijmehy6PsyN3eShCAv7Xh653mcOEThFmoj4/nrgUSfP63G6c1
         dxDy9uKDeEoNpX+VXHx3ejYfrWLP+QHvOc+U1Sx+fcOrsmN8FSMV503FxEuunLjFTCw2
         YoJw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:message-id:date:mime-version:user-agent:reply-to
         :subject:content-language:to:references:from:in-reply-to
         :content-transfer-encoding;
        bh=XX5rHighIskvO6S2Qrw71pxmwvjWU7Eftr8yjX/vn+s=;
        b=WH/TxfwiPHkGyAh+icarn/oJttKhH15VDAq94dgFiEjYZ2/nEsH+Qh4Tc/MZ+kw1FN
         Yr3prRYNsrZumMKZOZUw38Wrcy2B9Rxb0Q7R9IVz5+zcLOPSAyE80FDfEBGlPcBRYHCq
         RRMDNh3aUtBzlewsdBvkKVPVEIl/fDblpJs3ufMp/C06QcF5r4rw68vjRttJ0mN2vBtn
         2ywZ/SxY5yeHTKk+WnLEYpGxxJxu43Y49MIYsypGGWOgnxWU71H46VApGEyI6lVcEZ35
         WhLxOktv1eBDmgMn08fCeTAO8tE13zmYOxJ3Fk8tsGaC5o/i9yPiONHQIm8iErSCXytF
         aeTQ==
X-Gm-Message-State: AOAM532DMM34gW6HMN/CRx1cxlLBT93dTRU/VvlTEi3JPRibrGCkPgQl
	+/X8a3SLvZjcvIAy8yLMqPk=
X-Google-Smtp-Source: ABdhPJzrmbF9uBasaiXaGH1ywo7w/Llb/BcX9Vt6C4a5R8MOO43+CV1K5XLRUAR1JFRebtCrujCMmQ==
X-Received: by 2002:a05:600c:1e04:b0:38c:9b91:db07 with SMTP id ay4-20020a05600c1e0400b0038c9b91db07mr36365349wmb.55.1648478572200;
        Mon, 28 Mar 2022 07:42:52 -0700 (PDT)
Message-ID: <86c975b5-e75a-1c20-3e7a-d7f7c1780cc4@gmail.com>
Date: Mon, 28 Mar 2022 15:42:51 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.7.0
Reply-To: paul@xen.org
Subject: Re: TestSign cert passcodes
Content-Language: en-US
To: Owen Smith <owen.smith@citrix.com>, "paul@xen.org" <paul@xen.org>,
 "win-pv-devel@lists.xenproject.org" <win-pv-devel@lists.xenproject.org>
References: <DS7PR03MB5477FF7BF632E3D42F10559DFE1D9@DS7PR03MB5477.namprd03.prod.outlook.com>
 <39837734-86b4-e20b-d226-5d8535ceb9ef@gmail.com>
 <DS7PR03MB54771A52F9CE1FA229A5605CFE1D9@DS7PR03MB5477.namprd03.prod.outlook.com>
From: "Durrant, Paul" <xadimgnik@gmail.com>
In-Reply-To: <DS7PR03MB54771A52F9CE1FA229A5605CFE1D9@DS7PR03MB5477.namprd03.prod.outlook.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 28/03/2022 15:04, Owen Smith wrote:
> -----Original Message-----
> From: win-pv-devel <win-pv-devel-bounces@lists.xenproject.org> On Behalf Of Durrant, Paul
> Sent: 28 March 2022 14:22
> To: win-pv-devel@lists.xenproject.org
> Subject: Re: TestSign cert passcodes
> 
> [CAUTION - EXTERNAL EMAIL] DO NOT reply, click links, or open attachments unless you have verified the sender and know the content is safe.
> 
> On 28/03/2022 11:36, Owen Smith wrote:
>> Hi,
>>
>> I’ve been forced to set up a new dev environment, and am getting an
>> error trying to TestSign the binaries. I’m getting prompted for a
>> passcode for the pfx and don’t see this documented in the BUILD.md
>>
> 
> What happens if you just hit enter? There should be no passcode set.
> "error: Invalid certificate"
> 
> If I regenerate the pfx, it works and test signs the binaries
> 
> Comparing the regenerated pfx with the original, I can see no immediate differences (other than start date and serial number)
> 

That's really odd. I've never had an issue; but then it is about 2 years 
since I set up an env from scratch.

   Paul


From win-pv-devel-bounces@lists.xenproject.org Wed Mar 30 09:32:39 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Wed, 30 Mar 2022 09:32:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.296108.504015 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nZUgw-0008Ba-1O; Wed, 30 Mar 2022 09:32:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 296108.504015; Wed, 30 Mar 2022 09:32:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nZUgv-0008BT-TS; Wed, 30 Mar 2022 09:32:37 +0000
Received: by outflank-mailman (input) for mailman id 296108;
 Wed, 30 Mar 2022 09:32:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xR6P=UJ=citrix.com=prvs=0814e68e9=owen.smith@srs-se1.protection.inumbo.net>)
 id 1nZUgu-0008BN-6T
 for win-pv-devel@lists.xenproject.org; Wed, 30 Mar 2022 09:32:36 +0000
Received: from esa6.hc3370-68.iphmx.com (esa6.hc3370-68.iphmx.com
 [216.71.155.175]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5108ab79-b00c-11ec-8fbc-03012f2f19d4;
 Wed, 30 Mar 2022 11:32:32 +0200 (CEST)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5108ab79-b00c-11ec-8fbc-03012f2f19d4
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1648632751;
  h=from:to:cc:subject:date:message-id:mime-version:
   content-transfer-encoding;
  bh=BufDlqqdzwHuz+hLUNLREV8gFYp9rp3ovRgkvHmGm7M=;
  b=RH25zarSJ8Bd5pRCNypr2A0/JopRPl1Sza9pfI7tYquU6hZ5iGQuOtu9
   exNKTtou/TjWqNsxyqJhy4L1IEb9heok0wrRfPgStrNeIXaiTpaOujshi
   NYJtaoSuLEqCJdl2nvbpl7tZNsoZOgcuRyhGwMtdiERPJ+4hXY4VJJHMP
   U=;
Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
X-SBRS: 5.1
X-MesageID: 67426976
X-Ironport-Server: esa6.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:kRFWEqoKPC4jeecjhqm9mpX8rTxeBmJCZRIvgKrLsJaIsI4StFCzt
 garIBmGa6yJYGahc491aNzi9E9V6peDnNAwTgNqpSkxEXlE8ZuZCYyVIHmrMnLJJKUvbq7GA
 +byyDXkBJppJpMJjk71atANlVEliefQAOCU5NfsYkidfyc9IMsaoU8lyrZRbrJA24DjWVvW4
 ouq+aUzBXf+s9JKGjNMg068gEsHUMTa4Fv0aXRnOJinFHeH/5UkJMp3yZOZdhMUcaENdgKOf
 M7RzanRw4/s10xF5uVJMFrMWhZirrb6ZWBig5fNMkSoqkAqSicais7XOBeAAKv+Zvrgc91Zk
 b1wWZKMpQgBGJ3QpeQBVB5kI3tMAIdb1ueALWq+rpnGp6HGWyOEL/RGCUg3OcsT+/ptAHEI/
 vsdQNwPRknd3aTsmuv9E7QywJR4RCXoFNp3VnVIyi7UC7A9RJHfQ43B5MNC3Sd2jcdLdRrbT
 5RGNWAxPUueC/FJEmoKKpY0ofasv3/mfjJkpVHWiId0zXeGmWSd15CyaYGIK7RmX/59gUee4
 3rYumj0HBweHNie0iaetGKhgPfVmiH2U55UE6e3nsOGm3XKmDZVUkdPEwLm/7/p0SZSRu6zN
 WQyyjYTrbQ183CbFOvAdQCW+DmIrB8DDo84//IB1CmBza/d4gC8D2cCTyJcZNFOiPLaVQDGx
 XfSwYq3WGUHXKm9DCvEq+zK9W/a1T09dzdqWMMScecSDzAPSqkXhwmHcNtsGbXdYjbdSWCpm
 GDiQMTTatwuYS83O0eToAivb9GE/MGhousJCuP/BDjNAuRRPtPNWmBQwQKHhcus1a7AJrV7g
 FAKmtKF8McFBoyXmSqGTY0lRe/1t6vVamSG2QQ3Q/HNEghBHVb5J+i8BxkkeS9U3jssI2e1M
 Cc/RysPjHOsAJdaRfAuON/gYyjb5aPhCc7kRpjpgilmOfBMmPu81Hg2Pya4hjm1+GB1yP1XE
 crLIK6EUCdBYYw6nWXeegvo+eJyrszI7TiIHs6TItXO+ef2WUN5vp9YaQPXNbpotfnsTce82
 483CvZmAi53CIXWChQ7O6ZKRbzWBRDX3azLlvE=
IronPort-HdrOrdr: A9a23:aW3sy69RnLXtEh126vBuk+DSI+orL9Y04lQ7vn2YSXRuHfBw8P
 re+cjztCWE7wr5N0tApTntAsS9qBDnhPxICOsqXYtKNTOO0ADDEGgL1/qH/9SKIUPDH4BmuZ
 uIC5IOa+HNMQ==
X-IronPort-AV: E=Sophos;i="5.90,222,1643691600"; 
   d="scan'208";a="67426976"
From: Owen Smith <owen.smith@citrix.com>
To: <win-pv-devel@lists.xenproject.org>
CC: Owen Smith <owen.smith@citrix.com>
Subject: [PATCH] Refactor Wmi.c
Date: Wed, 30 Mar 2022 10:32:23 +0100
Message-ID: <20220330093223.659-1-owen.smith@citrix.com>
X-Mailer: git-send-email 2.32.0.windows.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

* Moves functions to be in related locations in file
* Formats code to appropriate code style
* Inlines some functions that are only called from 1 location
* Uses LIST_ENTRY macros to access linked lists

Signed-off-by: Owen Smith <owen.smith@citrix.com>
---
 src/xeniface/wmi.c | 3794 ++++++++++++++++++++++----------------------
 1 file changed, 1907 insertions(+), 1887 deletions(-)

diff --git a/src/xeniface/wmi.c b/src/xeniface/wmi.c
index 1b4c6c6..0d21b92 100644
--- a/src/xeniface/wmi.c
+++ b/src/xeniface/wmi.c
@@ -37,11 +37,11 @@
 #include <stdio.h>
 #include <guiddef.h>
 #define NTSTRSAFE_LIB
-#include<ntstrsafe.h>
+#include <ntstrsafe.h>
 #include "wmi.h"
 #include "driver.h"
-#include "..\..\include\store_interface.h"
-#include "..\..\include\suspend_interface.h"
+#include "store_interface.h"
+#include "suspend_interface.h"
 #include "log.h"
 #include "xeniface_ioctls.h"
 #include <version.h>
@@ -53,6 +53,54 @@
 #define UTF8MASK3 0x1FF800
 #define UTF8MASK4 0x1F0000
 
+typedef enum _WMI_TYPE {
+    WMI_DONE,
+    WMI_STRING,
+    WMI_BOOLEAN,
+    WMI_SINT8,
+    WMI_UINT8,
+    WMI_SINT16,
+    WMI_UINT16,
+    WMI_INT32,
+    WMI_UINT32,
+    WMI_SINT64,
+    WMI_UINT64,
+    WMI_DATETIME,
+    WMI_BUFFER,
+    WMI_OFFSET,
+    WMI_STRINGOFFSET
+} WMI_TYPE;
+
+#define MAX_WATCH_COUNT (MAXIMUM_WAIT_OBJECTS -1)
+
+typedef struct _XENSTORE_SESSION {
+    LIST_ENTRY                  ListEntry;
+    LONG                        SessionId;
+    UNICODE_STRING              StringId;
+    UNICODE_STRING              InstanceName;
+    PXENBUS_STORE_TRANSACTION   Transaction;
+    LIST_ENTRY                  WatchList;
+    int                         WatchCount;
+    PKEVENT                     WatchEvents[MAXIMUM_WAIT_OBJECTS];
+    KWAIT_BLOCK                 WatchWaitBlocks[MAXIMUM_WAIT_OBJECTS];
+    KEVENT                      SessionChangedEvent;
+    XENIFACE_MUTEX              WatchMapLock;
+    BOOLEAN                     Changed;
+    BOOLEAN                     Closing;
+    BOOLEAN                     Suspended;
+    PKTHREAD                    WatchThread;
+} XENSTORE_SESSION, *PXENSTORE_SESSION;
+
+typedef struct _XENSTORE_WATCH {
+    LIST_ENTRY                  ListEntry;
+    UNICODE_STRING              Path;
+    PXENIFACE_FDO               Fdo;
+    ULONG                       SuspendCount;
+    BOOLEAN                     Finished;
+    KEVENT                      WatchEvent;
+    PXENBUS_STORE_WATCH         WatchHandle;
+} XENSTORE_WATCH, *PXENSTORE_WATCH;
+
 static FORCEINLINE PVOID
 WmiAllocate(
     IN  ULONG   Length
@@ -70,19 +118,6 @@ WmiFree(
     __FreePoolWithTag(Buffer, WMI_POOL_TAG);
 }
 
-void LockSessions(
-        XENIFACE_FDO* fdoData)
-{
-    AcquireMutex(&fdoData->SessionLock);
-}
-
-
-void UnlockSessions(
-        XENIFACE_FDO* fdoData)
-{
-    ReleaseMutex(&fdoData->SessionLock);
-}
-
 // Rather inconveniently, xenstore needs UTF8 data, WMI works in UTF16
 // and windows doesn't provide conversion functions in any version
 // prior to Windows 7.
@@ -499,7 +534,7 @@ CloneUnicodeString(
 static NTSTATUS
 GetInstanceName(
     OUT PUNICODE_STRING dest,
-    IN  PXENIFACE_FDO   FdoData,
+    IN  PXENIFACE_FDO   Fdo,
     IN  const CHAR*     string
     )
 {
@@ -513,7 +548,7 @@ GetInstanceName(
     if (!NT_SUCCESS(status))
         goto fail1;
 
-    destsz = FdoData->SuggestedInstanceName.Length +
+    destsz = Fdo->SuggestedInstanceName.Length +
              sizeof(WCHAR) +
              unicode.Length;
 
@@ -524,7 +559,7 @@ GetInstanceName(
 
     status = RtlUnicodeStringPrintf(dest,
                                     L"%s\\%s",
-                                    FdoData->SuggestedInstanceName.Buffer,
+                                    Fdo->SuggestedInstanceName.Buffer,
                                     unicode.Buffer);
     if (!NT_SUCCESS(status))
         goto fail3;
@@ -544,7 +579,7 @@ fail1:
 
 static NTSTATUS
 WriteInstanceName(
-    IN  PXENIFACE_FDO   FdoData,
+    IN  PXENIFACE_FDO   Fdo,
     IN  const CHAR*     string,
     IN  PUCHAR          location
     )
@@ -552,7 +587,7 @@ WriteInstanceName(
     UNICODE_STRING      destination;
     NTSTATUS            status;
 
-    status = GetInstanceName(&destination, FdoData, string);
+    status = GetInstanceName(&destination, Fdo, string);
     if (!NT_SUCCESS(status))
         return status;
 
@@ -561,169 +596,209 @@ WriteInstanceName(
     return STATUS_SUCCESS;
 }
 
-typedef enum {
-    WMI_DONE,
-    WMI_STRING,
-    WMI_BOOLEAN,
-    WMI_SINT8,
-    WMI_UINT8,
-    WMI_SINT16,
-    WMI_UINT16,
-    WMI_INT32,
-    WMI_UINT32,
-    WMI_SINT64,
-    WMI_UINT64,
-    WMI_DATETIME,
-    WMI_BUFFER,
-    WMI_OFFSET,
-    WMI_STRINGOFFSET
-} WMI_TYPE;
+static PSTR
+Xmasprintf(
+    IN  const char* fmt,
+    ...
+    )
+{
+    va_list         argv;
+    PSTR            out;
+    size_t          basesize = 128;
+    size_t          unused;
+    NTSTATUS        status;
+
+    va_start(argv, fmt);
+    do {
+        basesize = basesize * 2;
+        out =  WmiAllocate((ULONG)basesize);
+        if (out == NULL)
+            return NULL;
+
+        status = RtlStringCbVPrintfExA(out, basesize, NULL, &unused, 0, fmt, argv);
+
+        WmiFree(out);
+    } while (status != STATUS_SUCCESS);
+
+    out = WmiAllocate((ULONG)(basesize - unused + 1));
+    if (out == NULL)
+        return NULL;
+
+    RtlStringCbVPrintfA(out, basesize - unused + 1, fmt, argv);
+
+    va_end(argv);
+    return out;
+}
+
+static FORCEINLINE VOID
+UnicodeShallowCopy(
+    IN  PUNICODE_STRING     dest,
+    IN  PUNICODE_STRING     src
+    )
+{
+    dest->Buffer = src->Buffer;
+    dest->Length = src->Length;
+    dest->MaximumLength = src->MaximumLength;
+}
+
+static FORCEINLINE int
+CompareUnicodeStrings(
+    IN  PCUNICODE_STRING    string1,
+    IN  PCUNICODE_STRING    string2
+    )
+{
+    if (string1->Length == string2->Length)
+        return RtlCompareMemory(string1->Buffer,
+                                string2->Buffer,
+                                string1->Length) != string1->Length;
+    return 1;
+}
+
+static int
+AccessWmiBuffer(
+    IN  PUCHAR  Buffer,
+    IN  int     readbuffer,
+    OUT ULONG*  RequiredSize,
+    IN  size_t  BufferSize,
+    ...
+    )
+{
+    va_list     vl;
+    ULONG_PTR   offset;
+    ULONG_PTR   offby;
+    PUCHAR      position = Buffer;
+    PUCHAR      endbuffer = Buffer + BufferSize;
+    int         overflow = 0;
 
-int AccessWmiBuffer(PUCHAR Buffer, int readbuffer, ULONG * RequiredSize,
-                    size_t BufferSize, ...) {
-    va_list vl;
-    ULONG_PTR offset;
-    ULONG_PTR offby;
-    PUCHAR position = Buffer;
-    PUCHAR endbuffer = Buffer + BufferSize;
-    int overflow=0;
     va_start(vl, BufferSize);
-    for(;;) {
+    for (;;) {
         WMI_TYPE type = va_arg(vl, WMI_TYPE);
-        if (type != WMI_DONE) {
-#define WMITYPECASE(_wmitype, _type, _align) \
-            case _wmitype: {\
-                _type** val; \
-                offby = ((ULONG_PTR)position)%(_align); \
-                offset = ((_align)-offby)%(_align) ; \
-                position += offset;\
-                if (position + sizeof(_type) > endbuffer) \
-                    overflow = TRUE;\
-                val = va_arg(vl, _type**); \
-                *val = NULL; \
-                if (!overflow) \
-                    *val = (_type *)position; \
-                position += sizeof(_type); } \
-                break
-            switch (type) {
-                case WMI_STRING:
-                    {
-                        UCHAR **countstr;
-                        USHORT strsize;
-                        offset = (2-((ULONG_PTR)position%2))%2;
-                        position+=offset;
-                        if (position + sizeof(USHORT) > endbuffer)
-                            overflow = TRUE;
-                        if (readbuffer) {
-                            if (!overflow)
-                                strsize = *(USHORT*)position;
-                            else
-                                strsize = 0;
-                            strsize+=sizeof(USHORT);
-                        }
-                        else {
-                            strsize = va_arg(vl, USHORT);
-                        }
-                        if (position + strsize  >endbuffer)
-                            overflow = TRUE;
-                        countstr = va_arg(vl, UCHAR**);
-                        *countstr = NULL;
-                        if (!overflow)
-                            *countstr = position;
-                        position +=strsize;
-                    }
-                    break;
-                case WMI_BUFFER:
-                    {
-                        ULONG size = va_arg(vl, ULONG);
-                        UCHAR **buffer;
-                        if (position + size > endbuffer)
-                            overflow = TRUE;
-                        buffer = va_arg(vl, UCHAR**);
-                        *buffer = NULL;
-                        if (!overflow)
-                            *buffer = position;
-                        position += size;
-                    }
-                    break;
-                case WMI_OFFSET:
-                    {
-                        ULONG inpos = va_arg(vl, ULONG);
-                        UCHAR *bufferpos = Buffer + inpos;
-                        ULONG insize = va_arg(vl, ULONG);
-                        UCHAR **writebuf = va_arg(vl, UCHAR**);
-                        *writebuf = NULL;
-                        if (bufferpos+ insize > endbuffer) {;
-                            overflow = TRUE;
-                        }
-                        else {
-                            *writebuf = bufferpos;
-                        }
-                        // Only update position if it extends
-                        // the required size of the buffer
-                        if (bufferpos+insize > position)
-                            position = bufferpos+insize;
-                    }
-                    break;
-                    case WMI_STRINGOFFSET:
-                    {
-                        UCHAR **countstr;
-                        USHORT strsize;
-                        ULONG inpos = va_arg(vl, ULONG);
-                        UCHAR *bufferpos = Buffer + inpos;
-                        if (bufferpos + sizeof(USHORT) > endbuffer)
-                            overflow = TRUE;
-                        if (readbuffer) {
-                            if (!overflow)
-                                strsize = *(USHORT*)bufferpos;
-                            else
-                                strsize = 0;
-                            strsize+=sizeof(USHORT);
-                        }
-                        else {
-                            strsize = va_arg(vl, USHORT);
-                        }
-                        if (bufferpos + strsize  >endbuffer)
-                            overflow = TRUE;
-                        countstr = va_arg(vl, UCHAR**);
-                        *countstr = NULL;
-                        if (!overflow)
-                            *countstr = bufferpos;
-                        if (bufferpos+strsize > position)
-                            position =bufferpos+strsize;
-                    }
-                    break;
-                WMITYPECASE(WMI_BOOLEAN, UCHAR, 1);
-                WMITYPECASE(WMI_SINT8, CHAR, 1);
-                WMITYPECASE(WMI_UINT8, UCHAR, 1);
-                WMITYPECASE(WMI_SINT16, SHORT, 2);
-                WMITYPECASE(WMI_UINT16, USHORT, 2);
-                WMITYPECASE(WMI_INT32, LONG, 4);
-                WMITYPECASE(WMI_UINT32, ULONG, 4);
-                WMITYPECASE(WMI_SINT64, LONGLONG, 8);
-                WMITYPECASE(WMI_UINT64, ULONGLONG, 8);
-                case WMI_DATETIME:
-                    {
-                        LPWSTR *val;
-                        offset = (2-((ULONG_PTR)position%2))%2;
-                        position += offset;
-                        if (position + sizeof(WCHAR)*25 > endbuffer)
-                            overflow = TRUE;
-                        val = va_arg(vl, LPWSTR*);
-                        *val = NULL;
-                        if (!overflow)
-                            *val = (LPWSTR )position;
-                        position += sizeof(WCHAR)*25;
-                    }
-                    break;
-                default:
-                    return FALSE;
-            }
-        }
-        else {
+        if (type == WMI_DONE)
             break;
+
+#define WMITYPECASE(_wmitype, _type, _align)            \
+        case _wmitype: {                                \
+            _type** val;                                \
+            offby = ((ULONG_PTR)position) % (_align);   \
+            offset = ((_align) - offby) % (_align);     \
+            position += offset;                         \
+            if (position + sizeof(_type) > endbuffer)   \
+                overflow = TRUE;                        \
+            val = va_arg(vl, _type**);                  \
+            *val = NULL;                                \
+            if (!overflow)                              \
+                *val = (_type *)position;               \
+            position += sizeof(_type);                  \
+        } break
+
+        switch (type) {
+        WMITYPECASE(WMI_BOOLEAN, UCHAR, 1);
+        WMITYPECASE(WMI_SINT8, CHAR, 1);
+        WMITYPECASE(WMI_UINT8, UCHAR, 1);
+        WMITYPECASE(WMI_SINT16, SHORT, 2);
+        WMITYPECASE(WMI_UINT16, USHORT, 2);
+        WMITYPECASE(WMI_INT32, LONG, 4);
+        WMITYPECASE(WMI_UINT32, ULONG, 4);
+        WMITYPECASE(WMI_SINT64, LONGLONG, 8);
+        WMITYPECASE(WMI_UINT64, ULONGLONG, 8);
+
+        case WMI_STRING: {
+            UCHAR** countstr;
+            USHORT  strsize;
+            offset = (2 - ((ULONG_PTR)position % 2)) % 2;
+            position += offset;
+            if (position + sizeof(USHORT) > endbuffer)
+                overflow = TRUE;
+            if (readbuffer) {
+                if (!overflow)
+                    strsize = *(USHORT*)position;
+                else
+                    strsize = 0;
+                strsize += sizeof(USHORT);
+            } else {
+                strsize = va_arg(vl, USHORT);
+            }
+            if (position + strsize > endbuffer)
+                overflow = TRUE;
+            countstr = va_arg(vl, UCHAR**);
+            *countstr = NULL;
+            if (!overflow)
+                *countstr = position;
+            position += strsize;
+        } break;
+
+        case WMI_BUFFER: {
+            ULONG   size = va_arg(vl, ULONG);
+            UCHAR** buffer;
+            if (position + size > endbuffer)
+                overflow = TRUE;
+            buffer = va_arg(vl, UCHAR**);
+            *buffer = NULL;
+            if (!overflow)
+                *buffer = position;
+            position += size;
+        } break;
+
+        case WMI_OFFSET: {
+            ULONG   inpos = va_arg(vl, ULONG);
+            UCHAR*  bufferpos = Buffer + inpos;
+            ULONG   insize = va_arg(vl, ULONG);
+            UCHAR** writebuf = va_arg(vl, UCHAR**);
+            *writebuf = NULL;
+            if (bufferpos + insize > endbuffer)
+                overflow = TRUE;
+            else
+                *writebuf = bufferpos;
+            // Only update position if it extends
+            // the required size of the buffer
+            if (bufferpos + insize > position)
+                position = bufferpos + insize;
+        } break;
+
+        case WMI_STRINGOFFSET: {
+            UCHAR** countstr;
+            USHORT  strsize;
+            ULONG   inpos = va_arg(vl, ULONG);
+            UCHAR*  bufferpos = Buffer + inpos;
+            if (bufferpos + sizeof(USHORT) > endbuffer)
+                overflow = TRUE;
+            if (readbuffer) {
+                if (!overflow)
+                    strsize = *(USHORT*)bufferpos;
+                else
+                    strsize = 0;
+                strsize += sizeof(USHORT);
+            } else {
+                strsize = va_arg(vl, USHORT);
+            }
+            if (bufferpos + strsize > endbuffer)
+                overflow = TRUE;
+            countstr = va_arg(vl, UCHAR**);
+            *countstr = NULL;
+            if (!overflow)
+                *countstr = bufferpos;
+            if (bufferpos + strsize > position)
+                position = bufferpos + strsize;
+        } break;
+
+        case WMI_DATETIME: {
+            LPWSTR* val;
+            offset = (2 - ((ULONG_PTR)position % 2)) % 2;
+            position += offset;
+            if (position + sizeof(WCHAR) * 25 > endbuffer)
+                overflow = TRUE;
+            val = va_arg(vl, LPWSTR*);
+            *val = NULL;
+            if (!overflow)
+                *val = (LPWSTR)position;
+            position += sizeof(WCHAR) * 25;
+        } break;
+
+        default:
+            return FALSE;
         }
     }
+
     *RequiredSize = (ULONG)(position - Buffer);
     va_end(vl);
     if (overflow)
@@ -731,1086 +806,922 @@ int AccessWmiBuffer(PUCHAR Buffer, int readbuffer, ULONG * RequiredSize,
     return TRUE;
 }
 
-#define MAX_WATCH_COUNT (MAXIMUM_WAIT_OBJECTS -1)
+static FORCEINLINE PXENSTORE_SESSION
+FindSessionLocked(
+    IN  PXENIFACE_FDO   Fdo,
+    IN  LONG            Id
+    )
+{
+    PLIST_ENTRY         ListEntry;
+    PXENSTORE_SESSION   Session;
 
-typedef struct _XenStoreSession {
-    LIST_ENTRY listentry;
-    LONG id;
-    UNICODE_STRING stringid;
-    UNICODE_STRING instancename;
-    PXENBUS_STORE_TRANSACTION transaction;
-    LIST_ENTRY watches;
-    int watchcount;
-    KEVENT* watchevents[MAXIMUM_WAIT_OBJECTS];
-    KWAIT_BLOCK watchwaitblockarray[MAXIMUM_WAIT_OBJECTS];
-    KEVENT SessionChangedEvent;
-    XENIFACE_MUTEX WatchMapLock;
-    BOOLEAN mapchanged;
-    BOOLEAN closing;
-    BOOLEAN suspended;
-    PKTHREAD WatchThread;
-} XenStoreSession;
-
-typedef struct _XenStoreWatch {
-    LIST_ENTRY listentry;
-    UNICODE_STRING path;
-    XENIFACE_FDO *fdoData;
-
-    ULONG   suspendcount;
-    BOOLEAN finished;
-    KEVENT watchevent;
-    PXENBUS_STORE_WATCH watchhandle;
-
-} XenStoreWatch;
-
-void UnicodeShallowCopy(UNICODE_STRING *dest, UNICODE_STRING *src) {
-    dest->Buffer = src->Buffer;
-    dest->Length = src->Length;
-    dest->MaximumLength = src->MaximumLength;
-}
+    ASSERT3P(Fdo->SessionLock.Owner, ==, KeGetCurrentThread());
 
+    for (ListEntry = Fdo->SessionHead.Flink;
+         ListEntry != &Fdo->SessionHead;
+         ListEntry = ListEntry->Flink) {
+        Session = CONTAINING_RECORD(ListEntry, XENSTORE_SESSION, ListEntry);
 
-XenStoreSession*
-FindSessionLocked(XENIFACE_FDO *fdoData,
-                                LONG id) {
-    XenStoreSession *session;
+        if (Session->SessionId != Id)
+            continue;
 
-    session = (XenStoreSession *)fdoData->SessionHead.Flink;
-    while (session != (XenStoreSession *)&fdoData->SessionHead){
-        if (session->id == id) {
-            if (session->suspended)
-                return NULL;
-            return session;
-        }
-        session = (XenStoreSession *)session->listentry.Flink;
+        return Session->Suspended ? NULL : Session;
     }
     return NULL;
 }
 
+static FORCEINLINE PXENSTORE_WATCH
+SessionFindWatchLocked(
+    IN  PXENSTORE_SESSION   Session,
+    IN  PUNICODE_STRING     Path
+    )
+{
+    PLIST_ENTRY             ListEntry;
+    PXENSTORE_WATCH         Watch;
 
-int CompareUnicodeStrings(PCUNICODE_STRING string1, PCUNICODE_STRING string2) {
-    if (string1->Length == string2->Length) {
-        return RtlCompareMemory(string1->Buffer,string2->Buffer, string1->Length) != string1->Length;
-    }
-    return 1;
-
-}
-
-XenStoreWatch *
-SessionFindWatchLocked(XenStoreSession *session,
-                        UNICODE_STRING *path) {
-    XenStoreWatch * watch;
+    ASSERT3P(Session->WatchMapLock.Owner, ==, KeGetCurrentThread());
 
-    Trace("Wait for session watch lock\n");
-    AcquireMutex(&session->WatchMapLock);
-    Trace("got session watch lock\n");
-    watch = (XenStoreWatch *)session->watches.Flink;
+    for (ListEntry = Session->WatchList.Flink;
+         ListEntry != &Session->WatchList;
+         ListEntry = ListEntry->Flink) {
+        Watch = CONTAINING_RECORD(ListEntry, XENSTORE_WATCH, ListEntry);
 
-    while (watch != (XenStoreWatch *)&session->watches){
-        if (CompareUnicodeStrings(path, &watch->path)==0) {
-            return watch;
-        }
-        watch = (XenStoreWatch *)watch->listentry.Flink;
+        if (CompareUnicodeStrings(Path, &Watch->Path) == 0)
+            return Watch;
     }
-
-    Warning("couldn't find watch\n");
     return NULL;
-
 }
 
-VOID
-WmiFireSuspendEvent(
-    IN  PXENIFACE_FDO   Fdo
+static VOID
+FireWatch(
+    IN  PXENSTORE_WATCH Watch
     )
 {
-    Info("Ready to unsuspend Event\n");
-    KeSetEvent(&Fdo->registryWriteEvent, IO_NO_INCREMENT, FALSE);
-
-    if (!Fdo->WmiReady)
-        return;
-
-    Trace("Fire Suspend Event\n");
-    WmiFireEvent(Fdo->Dx->DeviceObject,
-                 (LPGUID)&OBJECT_GUID(XenStoreUnsuspendedEvent),
-                 0,
-                 0,
-                 NULL);
-}
-
-void FireWatch(XenStoreWatch* watch) {
-    UCHAR * eventdata;
-    ULONG RequiredSize;
-    UCHAR *sesbuf;
+    UCHAR*              eventdata;
+    ULONG               RequiredSize;
+    UCHAR*              sesbuf;
 
-    (VOID) AccessWmiBuffer(0, FALSE, &RequiredSize, 0,
-            WMI_STRING, GetCountedUnicodeStringSize(&watch->path),
-                &sesbuf,
+    (VOID) AccessWmiBuffer(NULL, FALSE, &RequiredSize, 0,
+            WMI_STRING, GetCountedUnicodeStringSize(&Watch->Path), &sesbuf,
             WMI_DONE);
 
     eventdata = WmiAllocate(RequiredSize);
-    if (eventdata!=NULL) {
-        (VOID) AccessWmiBuffer(eventdata, FALSE, &RequiredSize, RequiredSize,
-            WMI_STRING, GetCountedUnicodeStringSize(&watch->path),
-                &sesbuf,
+    if (eventdata == NULL)
+        return;
+
+    (VOID) AccessWmiBuffer(eventdata, FALSE, &RequiredSize, RequiredSize,
+            WMI_STRING, GetCountedUnicodeStringSize(&Watch->Path), &sesbuf,
             WMI_DONE);
 
-        WriteCountedUnicodeString(&watch->path, sesbuf);
-    }
+    WriteCountedUnicodeString(&Watch->Path, sesbuf);
 
-    if (eventdata !=NULL) {
-        Trace("Fire Watch Event\n");
-        WmiFireEvent(watch->fdoData->Dx->DeviceObject,
-                     (LPGUID)&OBJECT_GUID(XenStoreWatchEvent),
-                     0,
-                     RequiredSize,
-                     eventdata);
-    }
+    Trace("Fire Watch Event\n");
+    WmiFireEvent(Watch->Fdo->Dx->DeviceObject,
+                 (LPGUID)&OBJECT_GUID(XenStoreWatchEvent),
+                 0,
+                 RequiredSize,
+                 eventdata);
 }
 
-
 KSTART_ROUTINE WatchCallbackThread;
-NTSTATUS
-StartWatch(XENIFACE_FDO *fdoData, XenStoreWatch *watch)
+
+static NTSTATUS
+StartWatch(
+    IN  PXENIFACE_FDO   Fdo,
+    IN  PXENSTORE_WATCH Watch
+    )
 {
-    char *tmppath;
-    ANSI_STRING ansipath;
-    NTSTATUS status;
-    status = RtlUnicodeStringToAnsiString(&ansipath, &watch->path, TRUE);
-    if (!NT_SUCCESS(status)) {
-        return STATUS_INSUFFICIENT_RESOURCES;
-    }
+    char*               tmppath;
+    ANSI_STRING         ansipath;
+    NTSTATUS            status;
+
+    status = RtlUnicodeStringToAnsiString(&ansipath, &Watch->Path, TRUE);
+    if (!NT_SUCCESS(status))
+        goto fail1;
+
     tmppath = WmiAllocate(ansipath.Length + 1);
-    if (!tmppath) {
-        RtlFreeAnsiString(&ansipath);
-        return STATUS_INSUFFICIENT_RESOURCES;
-    }
+    status = STATUS_INSUFFICIENT_RESOURCES;
+    if (tmppath == NULL)
+        goto fail2;
 
-    RtlCopyBytes(tmppath,ansipath.Buffer, ansipath.Length);
+    RtlCopyBytes(tmppath, ansipath.Buffer, ansipath.Length);
 
-    status = XENBUS_STORE(WatchAdd, &fdoData->StoreInterface, NULL, tmppath, &watch->watchevent, &watch->watchhandle );
-    if (!NT_SUCCESS(status)) {
-        WmiFree(tmppath);
-        RtlFreeAnsiString(&ansipath);
-        return status;
-    }
+    status = XENBUS_STORE(WatchAdd,
+                          &Fdo->StoreInterface,
+                          NULL,
+                          tmppath,
+                          &Watch->WatchEvent,
+                          &Watch->WatchHandle);
+    if (!NT_SUCCESS(status))
+        goto fail3;
 
-    Info("Start Watch %p\n", watch->watchhandle);
+    Info("Start Watch %p\n", Watch->WatchHandle);
 
     WmiFree(tmppath);
     RtlFreeAnsiString(&ansipath);
 
     return STATUS_SUCCESS;
-}
-
 
-VOID WatchCallbackThread(__in PVOID StartContext) {
-    NTSTATUS status;
-    int i=0;
-    XenStoreSession * session = (XenStoreSession*) StartContext;
+fail3:
+    WmiFree(tmppath);
+fail2:
+    RtlFreeAnsiString(&ansipath);
+fail1:
+    return status;
+}
 
-    for(;;) {
-        AcquireMutex(&session->WatchMapLock);
-        if (session->mapchanged) {
+VOID
+WatchCallbackThread(
+    __in PVOID          StartContext
+    )
+{
+    NTSTATUS            status;
+    PLIST_ENTRY         ListEntry;
+    PXENSTORE_WATCH     Watch;
+    PXENSTORE_SESSION   Session = (PXENSTORE_SESSION)StartContext;
+    int                 Count = 0;
+
+    for (;;) {
+        AcquireMutex(&Session->WatchMapLock);
+        if (Session->Changed) {
             // Construct a new mapping
-            XenStoreWatch *watch;
             Trace("Construct a new mapping\n");
-            watch = (XenStoreWatch *)session->watches.Flink;
-            for (i=0; watch != (XenStoreWatch *)&session->watches; i++) {
-                session->watchevents[i] = &watch->watchevent;
-                watch = (XenStoreWatch *)watch->listentry.Flink;
+            for (Count = 0, ListEntry = Session->WatchList.Flink;
+                 ListEntry != &Session->WatchList;
+                 Count++, ListEntry = ListEntry->Flink) {
+                Watch = CONTAINING_RECORD(ListEntry, XENSTORE_WATCH, ListEntry);
+
+                Session->WatchEvents[Count] = &Watch->WatchEvent;
             }
-            session->mapchanged = FALSE;
-            session->watchevents[i] = &session->SessionChangedEvent;
+            Session->WatchEvents[Count] = &Session->SessionChangedEvent;
+            Session->Changed = FALSE;
         }
-        ReleaseMutex(&session->WatchMapLock);
+        ReleaseMutex(&Session->WatchMapLock);
+
         Trace("Wait for new event\n");
-        status = KeWaitForMultipleObjects(i+1, session->watchevents, WaitAny, Executive, KernelMode, TRUE, NULL, session->watchwaitblockarray);
+        status = KeWaitForMultipleObjects(Count + 1,
+                                          Session->WatchEvents,
+                                          WaitAny,
+                                          Executive,
+                                          KernelMode,
+                                          TRUE,
+                                          NULL,
+                                          Session->WatchWaitBlocks);
         Trace("got new event\n");
-        if ((status >= STATUS_WAIT_0) && (status < STATUS_WAIT_0 +i )) {
-            XenStoreWatch *watch;
+
+        if ((status >= STATUS_WAIT_0) && (status < STATUS_WAIT_0 + Count)) {
             Trace("watch or suspend\n");
-            watch = CONTAINING_RECORD(session->watchevents[status-STATUS_WAIT_0], XenStoreWatch, watchevent );
-            AcquireMutex(&session->WatchMapLock);
-            KeClearEvent(&watch->watchevent);
-
-
-            if (watch->finished) {
-                FreeUnicodeStringBuffer(&watch->path);
-                RemoveEntryList((LIST_ENTRY*)watch);
-                WmiFree(watch);
-                session->mapchanged = TRUE;
-                session->watchcount --;
-            } else if (!session->suspended &&
-                       watch->suspendcount != XENBUS_SUSPEND(GetCount, &watch->fdoData->SuspendInterface)) {
-                watch->suspendcount = XENBUS_SUSPEND(GetCount, &watch->fdoData->SuspendInterface);
-                Info("SessionSuspendResumeUnwatch %p\n", watch->watchhandle);
-
-                XENBUS_STORE(WatchRemove, &watch->fdoData->StoreInterface, watch->watchhandle);
-                watch->watchhandle = NULL;
-                StartWatch(watch->fdoData, watch);
+            Watch = CONTAINING_RECORD(Session->WatchEvents[status - STATUS_WAIT_0],
+                                      XENSTORE_WATCH,
+                                      WatchEvent);
+
+            AcquireMutex(&Session->WatchMapLock);
+            KeClearEvent(&Watch->WatchEvent);
+
+            if (Watch->Finished) {
+                FreeUnicodeStringBuffer(&Watch->Path);
+                RemoveEntryList(&Watch->ListEntry);
+                WmiFree(Watch);
+
+                Session->Changed = TRUE;
+                Session->WatchCount--;
+            } else if (!Session->Suspended &&
+                       Watch->SuspendCount != XENBUS_SUSPEND(GetCount, &Watch->Fdo->SuspendInterface)) {
+                Watch->SuspendCount = XENBUS_SUSPEND(GetCount, &Watch->Fdo->SuspendInterface);
+                Info("SessionSuspendResumeUnwatch %p\n", Watch->WatchHandle);
+
+                XENBUS_STORE(WatchRemove, &Watch->Fdo->StoreInterface, Watch->WatchHandle);
+                Watch->WatchHandle = NULL;
+                StartWatch(Watch->Fdo, Watch);
             } else {
-                FireWatch(watch);
+                FireWatch(Watch);
             }
-            ReleaseMutex(&session->WatchMapLock);
-        }
-        else if ( status == STATUS_WAIT_0 + i) {
-            AcquireMutex(&session->WatchMapLock);
-            KeClearEvent(&session->SessionChangedEvent);
-            if (session->closing==TRUE) {
+
+            ReleaseMutex(&Session->WatchMapLock);
+        } else if (status == STATUS_WAIT_0 + Count) {
+            AcquireMutex(&Session->WatchMapLock);
+            KeClearEvent(&Session->SessionChangedEvent);
+            if (Session->Closing) {
                 Trace("Trying to end session thread\n");
-                if (session->watchcount != 0) {
-                    XenStoreWatch *watch;
-                    for (watch = (XenStoreWatch *)session->watches.Flink;
-                        watch!=(XenStoreWatch *)&session->watches;
-                        watch=(XenStoreWatch *)session->watches.Flink) {
-                            FreeUnicodeStringBuffer(&watch->path);
-                            RemoveEntryList((LIST_ENTRY*)watch);
-                            WmiFree(watch);
-                            session->mapchanged = TRUE;
-                            session->watchcount --;
-                    }
+                while (!IsListEmpty(&Session->WatchList)) {
+                    ListEntry = RemoveHeadList(&Session->WatchList);
+                    ASSERT(ListEntry != &Session->WatchList);
+
+                    Session->WatchCount--;
+                    Session->Changed = TRUE;
+
+                    Watch = CONTAINING_RECORD(ListEntry, XENSTORE_WATCH, ListEntry);
+
+                    FreeUnicodeStringBuffer(&Watch->Path);
+                    WmiFree(Watch);
                 }
-                ReleaseMutex(&session->WatchMapLock);
+                ReleaseMutex(&Session->WatchMapLock);
+
                 Trace("Ending session thread\n");
                 PsTerminateSystemThread(STATUS_SUCCESS);
-                //ReleaseMutex(&session->WatchMapLock);
-            }
-            else {
-
-                ReleaseMutex(&session->WatchMapLock);
+            } else {
+                ReleaseMutex(&Session->WatchMapLock);
             }
         }
-
     }
 }
 
-NTSTATUS
-SessionAddWatchLocked(XenStoreSession *session,
-                        XENIFACE_FDO* fdoData,
-                        UNICODE_STRING *path,
-                        XenStoreWatch **watch) {
-
-
-    NTSTATUS status;
-    XenStoreWatch *pwatch;
-
-    if (session->watchcount >= MAX_WATCH_COUNT) {
-        return STATUS_INSUFFICIENT_RESOURCES;
-    }
-
-    *watch = WmiAllocate(sizeof(XenStoreWatch));
-    if (*watch == NULL)
-        return STATUS_INSUFFICIENT_RESOURCES;
-
-    (*watch)->finished = FALSE;
-    (*watch)->fdoData = fdoData;
-    UnicodeShallowCopy(&(*watch)->path, path);
-
+static NTSTATUS
+SessionAddWatchLocked(
+    IN  PXENIFACE_FDO       Fdo,
+    IN  PXENSTORE_SESSION   Session,
+    IN  PUNICODE_STRING     Path
+    )
+{
+    PXENSTORE_WATCH         Watch;
+    NTSTATUS                status;
 
+    ASSERT3P(Session->WatchMapLock.Owner, ==, KeGetCurrentThread());
 
-    (*watch)->suspendcount = XENBUS_SUSPEND(GetCount, &fdoData->SuspendInterface);
+    status = STATUS_INSUFFICIENT_RESOURCES;
+    if (Session->WatchCount >= MAX_WATCH_COUNT)
+        goto fail1;
 
+    Watch = WmiAllocate(sizeof(XENSTORE_WATCH));
+    status = STATUS_INSUFFICIENT_RESOURCES;
+    if (Watch == NULL)
+        goto fail2;
 
-    KeInitializeEvent(&(*watch)->watchevent, NotificationEvent, FALSE);
+    Watch->Finished = FALSE;
+    Watch->Fdo = Fdo;
+    Watch->SuspendCount = XENBUS_SUSPEND(GetCount, &Fdo->SuspendInterface);
 
+    UnicodeShallowCopy(&Watch->Path, Path);
+    KeInitializeEvent(&Watch->WatchEvent, NotificationEvent, FALSE);
 
-    status = StartWatch(fdoData, *watch);
-    if ((!NT_SUCCESS(status)) || ((*watch)->watchhandle == NULL)) {
-        WmiFree(*watch);
-        return STATUS_INSUFFICIENT_RESOURCES;
-    }
+    status = StartWatch(Fdo, Watch);
+    if (!NT_SUCCESS(status))
+        goto fail3;
 
-    AcquireMutex(&session->WatchMapLock);
-    session->mapchanged = TRUE;
-    KeSetEvent(&session->SessionChangedEvent, IO_NO_INCREMENT,FALSE);
-    session->watchcount++;
-    InsertHeadList(&session->watches,(PLIST_ENTRY)(*watch));
+    ASSERT(Watch->WatchHandle != NULL);
 
-    Trace("WATCHLIST for session %p-----------\n", session);
-    pwatch = (XenStoreWatch *)session->watches.Flink;
+    Session->WatchCount++;
+    InsertHeadList(&Session->WatchList, &Watch->ListEntry);
 
-    while (pwatch != (XenStoreWatch *)&session->watches){
-        Trace("WATCHLIST %p\n", pwatch->watchhandle);
-        pwatch = (XenStoreWatch *)pwatch->listentry.Flink;
-    }
-    Trace("WATCHLIST-------------------\n");
+    Session->Changed = TRUE;
+    KeSetEvent(&Session->SessionChangedEvent, IO_NO_INCREMENT,FALSE);
 
-    ReleaseMutex(&session->WatchMapLock);
     return STATUS_SUCCESS;
 
+fail3:
+    WmiFree(Watch);
+fail2:
+fail1:
+    return status;
 }
 
-void SessionRemoveWatchLocked(XenStoreSession *session, XenStoreWatch *watch) {
-
-    XenStoreWatch *pwatch;
-    Trace("Remove watch locked\n");
-    Trace("watch %p\n", watch);
-    Trace("handle %p\n", watch->watchhandle);
+static VOID
+SessionRemoveWatchLocked(
+    IN  PXENSTORE_WATCH     Watch
+    )
+{
+    // ASSERT3P(Session->WatchMapLock.Owner, ==, KeGetCurrentThread());
 
-    if (watch->watchhandle) {
-        XENBUS_STORE(WatchRemove, &watch->fdoData->StoreInterface, watch->watchhandle);
-        watch->watchhandle=NULL;
-        watch->finished = TRUE;
-        Trace("WATCHLIST for session %p-----------\n", session);
-    pwatch = (XenStoreWatch *)session->watches.Flink;
+    if (Watch->WatchHandle)
+        XENBUS_STORE(WatchRemove, &Watch->Fdo->StoreInterface, Watch->WatchHandle);
+    Watch->WatchHandle = NULL;
 
-    while (pwatch != (XenStoreWatch *)&session->watches){
-        Trace("WATCHLIST %p\n", pwatch->watchhandle);
-        pwatch = (XenStoreWatch *)pwatch->listentry.Flink;
-    }
-    Trace("WATCHLIST-------------------\n");
-        KeSetEvent(&watch->watchevent, IO_NO_INCREMENT,FALSE);
-    }
+    Watch->Finished = TRUE;
 
+    KeSetEvent(&Watch->WatchEvent, IO_NO_INCREMENT,FALSE);
 }
 
-void SessionRemoveWatchesLocked(XenStoreSession *session) {
-    XenStoreWatch *watch;
-
-    Trace("wait remove mutex\n");
-    AcquireMutex(&session->WatchMapLock);
-    for (watch = (XenStoreWatch *)session->watches.Flink;
-         watch!=(XenStoreWatch *)&session->watches;
-         watch=(XenStoreWatch *)watch->listentry.Flink) {
+static PXENSTORE_SESSION
+FindSessionByInstanceLocked(
+    IN  PXENIFACE_FDO       Fdo,
+    IN  PUNICODE_STRING     Instance
+    )
+{
+    PLIST_ENTRY             ListEntry;
+    PXENSTORE_SESSION       Session;
 
-        Trace("try remove %p\n", session->watches.Flink);
-        SessionRemoveWatchLocked(session, watch);
-    }
-    Trace("release remove mutex\n");
-    ReleaseMutex(&session->WatchMapLock);
-}
+    ASSERT3P(Fdo->SessionLock.Owner, ==, KeGetCurrentThread());
 
+    for (ListEntry = Fdo->SessionHead.Flink;
+         ListEntry != &Fdo->SessionHead;
+         ListEntry = ListEntry->Flink) {
+        Session = CONTAINING_RECORD(ListEntry, XENSTORE_SESSION, ListEntry);
 
-XenStoreSession*
-FindSessionByInstanceLocked(XENIFACE_FDO *fdoData,
-                            UNICODE_STRING *instance) {
-    XenStoreSession *session;
+        if (CompareUnicodeStrings(Instance, &Session->InstanceName) != 0)
+            continue;
 
-    session = (XenStoreSession *)fdoData->SessionHead.Flink;
-    while (session != (XenStoreSession *)&fdoData->SessionHead) {
-        if (CompareUnicodeStrings(instance, &session->instancename)==0) {
-            if (session->suspended)
-                return NULL;
-            return session;
-        }
-        session = (XenStoreSession *)session->listentry.Flink;
+        return Session->Suspended ? NULL : Session;
     }
     return NULL;
 }
 
-
 __checkReturn
-__success(return!=NULL)
-XenStoreSession *
-FindSessionByInstanceAndLock(XENIFACE_FDO *fdoData,
-                                UNICODE_STRING *instance) {
-    XenStoreSession *session;
-    LockSessions(fdoData);
-    session = FindSessionByInstanceLocked(fdoData, instance);
-    if (session == NULL) {
-         UnlockSessions(fdoData);
-    }
-    return session;
+__success(return != NULL)
+static PXENSTORE_SESSION
+FindSessionByInstanceAndLock(
+    IN  PXENIFACE_FDO   Fdo,
+    IN  PUNICODE_STRING Instance
+    )
+{
+    PXENSTORE_SESSION   Session;
+
+    AcquireMutex(&Fdo->SessionLock);
+    Session = FindSessionByInstanceLocked(Fdo, Instance);
+    if (Session == NULL)
+         ReleaseMutex(&Fdo->SessionLock);
+    return Session;
 }
 
-PSTR Xmasprintf(const char *fmt, ...) {
-    va_list argv;
-    PSTR out;
-    size_t basesize = 128;
-    size_t unused;
-    NTSTATUS status;
-    va_start(argv, fmt);
-    do{
-        basesize = basesize * 2;
-        out =  WmiAllocate((ULONG)basesize);
-        if (out == NULL)
-            return NULL;
+static NTSTATUS
+SessionCreate(
+    IN  PXENIFACE_FDO       Fdo,
+    IN  PUNICODE_STRING     StringId,
+    OUT ULONG*              SessionId
+    )
+{
+    PXENSTORE_SESSION       Session;
+    PSTR                    iname;
+    NTSTATUS                status;
+    ANSI_STRING             ansi;
+    HANDLE                  hthread;
+    OBJECT_ATTRIBUTES       oa;
+    int                     count = 0;
 
-        status = RtlStringCbVPrintfExA(out, basesize, NULL, &unused,0, fmt, argv);
+    status = STATUS_INSUFFICIENT_RESOURCES;
+    if (Fdo->Sessions == MAX_SESSIONS)
+        goto fail1;
 
-        WmiFree(out);
-    }while (status != STATUS_SUCCESS);
+    Session = WmiAllocate(sizeof(XENSTORE_SESSION));
+    status = STATUS_INSUFFICIENT_RESOURCES;
+    if (Session == NULL)
+        goto fail2;
 
-    out = WmiAllocate((ULONG)(basesize - unused + 1));
-    if (out == NULL)
-        return NULL;
+    status = RtlUnicodeStringToAnsiString(&ansi, StringId, TRUE);
+    if (!NT_SUCCESS(status))
+        goto fail3;
 
-    RtlStringCbVPrintfA(out, basesize-unused+1, fmt, argv);
+    InitializeMutex(&Session->WatchMapLock);
+    Session->Changed = TRUE;
 
-    va_end(argv);
-    return out;
-}
-
-NTSTATUS
-CreateNewSession(XENIFACE_FDO *fdoData,
-                    UNICODE_STRING *stringid,
-                    ULONG *sessionid) {
-    XenStoreSession *session;
-    PSTR iname;
-    NTSTATUS status;
-    ANSI_STRING ansi;
-    HANDLE hthread;
-    int count = 0;
-    OBJECT_ATTRIBUTES oa;
-    if (fdoData->Sessions == MAX_SESSIONS) {
-        return STATUS_INSUFFICIENT_RESOURCES;
-    }
-    session = WmiAllocate(sizeof(XenStoreSession));
-    if (session == NULL)
-        return STATUS_INSUFFICIENT_RESOURCES;
-
-    InitializeMutex(&session->WatchMapLock);
-    session->mapchanged = TRUE;
-    status = RtlUnicodeStringToAnsiString(&ansi, stringid, TRUE);
-    if (!NT_SUCCESS(status)) {
-        WmiFree(session);
-        return status;
-    }
-    LockSessions(fdoData);
-    do {
-        FreeUnicodeStringBuffer(&session->instancename);
-        iname = Xmasprintf("Session_%s_%d", ansi.Buffer, count);
+    AcquireMutex(&Fdo->SessionLock);
+    do {
+        FreeUnicodeStringBuffer(&Session->InstanceName);
+        iname = Xmasprintf("Session_%s_%d", ansi.Buffer, count);
 
         status = STATUS_NO_MEMORY;
-        if (iname == NULL) {
-            UnlockSessions(fdoData);
-            RtlFreeAnsiString(&ansi);
-            WmiFree(session);
-            return status;
-        }
+        if (iname == NULL)
+            goto fail4;
 
-        status = GetInstanceName(&session->instancename ,fdoData,iname);
+        status = GetInstanceName(&Session->InstanceName, Fdo, iname);
         WmiFree(iname);
-        if (!NT_SUCCESS(status)) {
-            UnlockSessions(fdoData);
-            RtlFreeAnsiString(&ansi);
-            WmiFree(session);
-            return status;
-        }
-        count++;
-
-    } while (FindSessionByInstanceLocked(fdoData, &session->instancename) != NULL);
-
+        if (!NT_SUCCESS(status))
+            goto fail5;
 
+        count++;
+    } while (FindSessionByInstanceLocked(Fdo, &Session->InstanceName) != NULL);
 
+    if (IsListEmpty(&Fdo->SessionHead)) {
+        Session->SessionId = 0;
+    } else {
+        Session->SessionId = ((PXENSTORE_SESSION)(Fdo->SessionHead.Flink))->SessionId + 1;
+        while (FindSessionLocked(Fdo, Session->SessionId))
+            Session->SessionId = (Session->SessionId + 1) % MAX_SESSIONS;
+    }
 
+    Session->Closing = FALSE;
+    Session->Transaction = NULL;
 
-    if (fdoData->SessionHead.Flink==&fdoData->SessionHead) {
-        session->id=0;
-    }
-    else {
-        session->id =((XenStoreSession*)(fdoData->SessionHead.Flink))->id+1;
-        while (FindSessionLocked(fdoData, session->id))
-            session->id = (session->id + 1) % MAX_SESSIONS;
-    }
-    session->transaction=NULL;
-    InsertHeadList((PLIST_ENTRY)&fdoData->SessionHead, (PLIST_ENTRY)session);
-    *sessionid = session->id;
-    UnicodeShallowCopy(&session->stringid, stringid);
+    *SessionId = Session->SessionId;
 
-    InitializeListHead((PLIST_ENTRY)&session->watches);
+    UnicodeShallowCopy(&Session->StringId, StringId);
+    InitializeListHead(&Session->WatchList);
+    KeInitializeEvent(&Session->SessionChangedEvent, NotificationEvent, FALSE);
 
-    KeInitializeEvent(&session->SessionChangedEvent, NotificationEvent, FALSE);
-    session->closing = FALSE;
-    if (fdoData->InterfacesAcquired){
+    if (Fdo->InterfacesAcquired){
         Trace("Add session unsuspended\n");
-        session->suspended=FALSE;
-    }
-    else {
+        Session->Suspended = FALSE;
+    } else {
         Trace("Add session suspended\n");
-        session->suspended=TRUE;
+        Session->Suspended = TRUE;
     }
-    fdoData->Sessions++;
+
+    InsertHeadList(&Fdo->SessionHead, &Session->ListEntry);
+    Fdo->Sessions++;
+
     InitializeObjectAttributes(&oa, NULL, OBJ_KERNEL_HANDLE, NULL, NULL);
 
-    status = PsCreateSystemThread(&hthread, THREAD_ALL_ACCESS, &oa, NULL, NULL, WatchCallbackThread, session);
-    if (!NT_SUCCESS(status)) {
-            RtlFreeAnsiString(&ansi);
-            WmiFree(session);
-            return status;
-    }
-    ObReferenceObjectByHandle(hthread, THREAD_ALL_ACCESS, NULL, KernelMode,  &session->WatchThread, NULL);
-    UnlockSessions(fdoData);
-    RtlFreeAnsiString(&ansi);
-    return STATUS_SUCCESS;
-}
+    status = PsCreateSystemThread(&hthread, THREAD_ALL_ACCESS, &oa, NULL, NULL, WatchCallbackThread, Session);
+    if (!NT_SUCCESS(status))
+        goto fail6;
 
-void
-RemoveSessionLocked(XENIFACE_FDO *fdoData,
-                    XenStoreSession *session) {
+    ObReferenceObjectByHandle(hthread, THREAD_ALL_ACCESS, NULL, KernelMode,  &Session->WatchThread, NULL);
+    ReleaseMutex(&Fdo->SessionLock);
 
-    Trace("RemoveSessionLocked\n");
-    RemoveEntryList((LIST_ENTRY*)session);
-    fdoData->Sessions--;
-    SessionRemoveWatchesLocked(session);
-    if (session->transaction != NULL) {
-        XENBUS_STORE(TransactionEnd, &fdoData->StoreInterface, session->transaction, FALSE);
-        session->transaction = NULL;
-    }
-    session->closing = TRUE;
-    KeSetEvent(&session->SessionChangedEvent, IO_NO_INCREMENT, FALSE);
-    KeWaitForSingleObject(session->WatchThread, Executive, KernelMode, FALSE, NULL);
-    ObDereferenceObject(session->WatchThread);
-    FreeUnicodeStringBuffer(&session->stringid);
-    FreeUnicodeStringBuffer(&session->instancename);
-    WmiFree(session);
-}
-
-void
-RemoveSession(XENIFACE_FDO *fdoData,
-                    XenStoreSession *session) {
-    Trace("RemoveSession\n");
-    LockSessions(fdoData);
-    RemoveSessionLocked(fdoData, session);
-    UnlockSessions(fdoData);
-}
-
-void SessionsRemoveAll(XENIFACE_FDO *fdoData) {
-    LockSessions(fdoData);
-    while (fdoData->SessionHead.Flink != &fdoData->SessionHead) {
-        RemoveSessionLocked(fdoData, (XenStoreSession *)fdoData->SessionHead.Flink);
-    }
-    UnlockSessions(fdoData);
-}
+    RtlFreeAnsiString(&ansi);
 
+    return STATUS_SUCCESS;
 
+fail6:
+    RemoveEntryList(&Session->ListEntry);
+    Fdo->Sessions--;
+fail5:
+fail4:
+    ReleaseMutex(&Fdo->SessionLock);
+    RtlFreeAnsiString(&ansi);
+fail3:
+    WmiFree(Session);
+fail2:
+fail1:
+    return status;
+}
 
-void SessionUnwatchWatchesLocked(XenStoreSession *session)
+static VOID
+SessionRemoveLocked(
+    IN  PXENIFACE_FDO       Fdo,
+    IN  PXENSTORE_SESSION   Session
+    )
 {
-    int i;
-    XenStoreWatch *watch;
-    AcquireMutex(&session->WatchMapLock);
-    watch = (XenStoreWatch *)session->watches.Flink;
-    for (i=0; watch != (XenStoreWatch *)&session->watches; i++) {
-        Trace("Suspend unwatch %p\n", watch->watchhandle);
+    PLIST_ENTRY             ListEntry;
+    PXENSTORE_WATCH         Watch;
 
-        XENBUS_STORE(WatchRemove, &watch->fdoData->StoreInterface, watch->watchhandle);
-        watch->watchhandle = NULL;
-        watch = (XenStoreWatch *)watch->listentry.Flink;
-    }
-    Trace("WATCHLIST for session %p-----------\n",session);
-    watch = (XenStoreWatch *)session->watches.Flink;
+    ASSERT3P(Fdo->SessionLock.Owner, ==, KeGetCurrentThread());
 
-    while (watch != (XenStoreWatch *)&session->watches){
-        Trace("WATCHLIST %p\n",watch->watchhandle);
-        watch = (XenStoreWatch *)watch->listentry.Flink;
-    }
-    Trace("WATCHLIST-------------------\n");
-    session->suspended=1;
-    ReleaseMutex(&session->WatchMapLock);
-}
+    RemoveEntryList(&Session->ListEntry);
+    Fdo->Sessions--;
 
-void SuspendSessionLocked(XENIFACE_FDO *fdoData,
-                         XenStoreSession *session) {
-    SessionUnwatchWatchesLocked(session);
-    if (session->transaction != NULL) {
-        Trace("End transaction %p\n",session->transaction);
+    AcquireMutex(&Session->WatchMapLock);
+    for (ListEntry = Session->WatchList.Flink;
+         ListEntry != &Session->WatchList;
+         ListEntry = ListEntry->Flink) {
+        Watch = CONTAINING_RECORD(ListEntry, XENSTORE_WATCH, ListEntry);
 
-        XENBUS_STORE(TransactionEnd, &fdoData->StoreInterface, session->transaction, FALSE);
-        session->transaction = NULL;
+        SessionRemoveWatchLocked(Watch);
     }
-}
-
-VOID
-WmiSessionsSuspendAll(
-    IN  PXENIFACE_FDO   Fdo
-    )
-{
-    XenStoreSession *session;
+    ReleaseMutex(&Session->WatchMapLock);
 
-    LockSessions(Fdo);
-    Trace("Suspend all sessions\n");
-    session = (XenStoreSession *)Fdo->SessionHead.Flink;
-    while (session != (XenStoreSession *)&Fdo->SessionHead) {
-        SuspendSessionLocked(Fdo, session);
-        session = (XenStoreSession *)session->listentry.Flink;
-    }
-    UnlockSessions(Fdo);
-}
+    if (Session->Transaction != NULL)
+        XENBUS_STORE(TransactionEnd, &Fdo->StoreInterface, Session->Transaction, FALSE);
+    Session->Transaction = NULL;
 
-void SessionRenewWatchesLocked(XenStoreSession *session) {
-    int i;
-    XenStoreWatch *watch;
-    AcquireMutex(&session->WatchMapLock);
-    watch = (XenStoreWatch *)session->watches.Flink;
-    for (i=0; watch != (XenStoreWatch *)&session->watches; i++) {
-        if (!watch->finished) {
-            watch->suspendcount = XENBUS_SUSPEND(GetCount, &watch->fdoData->SuspendInterface);
-            StartWatch(watch->fdoData, watch);
-        }
-        watch = (XenStoreWatch *)watch->listentry.Flink;
-    }
-    Trace("WATCHLIST for session %p-----------\n",session);
-    watch = (XenStoreWatch *)session->watches.Flink;
+    Session->Closing = TRUE;
 
-    while (watch != (XenStoreWatch *)&session->watches){
-        Trace("WATCHLIST %p\n",watch->watchhandle);
-        watch = (XenStoreWatch *)watch->listentry.Flink;
-    }
-    Trace("WATCHLIST-------------------\n");
-    session->suspended=0;
-    session->mapchanged = TRUE;
-    KeSetEvent(&session->SessionChangedEvent, IO_NO_INCREMENT,FALSE);
-    ReleaseMutex(&session->WatchMapLock);
-}
+    KeSetEvent(&Session->SessionChangedEvent, IO_NO_INCREMENT, FALSE);
+    KeWaitForSingleObject(Session->WatchThread, Executive, KernelMode, FALSE, NULL);
 
-void ResumeSessionLocked(XENIFACE_FDO *fdoData,
-                         XenStoreSession *session) {
-    SessionRenewWatchesLocked(session);
+    ObDereferenceObject(Session->WatchThread);
+    FreeUnicodeStringBuffer(&Session->StringId);
+    FreeUnicodeStringBuffer(&Session->InstanceName);
+    WmiFree(Session);
 }
 
-VOID
-WmiSessionsResumeAll(
+static VOID
+SessionsRemoveAll(
     IN  PXENIFACE_FDO   Fdo
     )
 {
-    XenStoreSession *session;
+    PXENSTORE_SESSION   Session;
 
-    LockSessions(Fdo);
-    Trace("Resume all sessions\n");
-    session = (XenStoreSession *)Fdo->SessionHead.Flink;
-    while (session != (XenStoreSession *)&Fdo->SessionHead) {
-        ResumeSessionLocked(Fdo, session);
-        session = (XenStoreSession *)session->listentry.Flink;
+    AcquireMutex(&Fdo->SessionLock);
+    while (!IsListEmpty(&Fdo->SessionHead)) {
+        ASSERT(Fdo->SessionHead.Flink != &Fdo->SessionHead);
+
+        Session = CONTAINING_RECORD(Fdo->SessionHead.Flink,
+                                    XENSTORE_SESSION,
+                                    ListEntry);
+
+        SessionRemoveLocked(Fdo, Session);
     }
-    UnlockSessions(Fdo);
+    ReleaseMutex(&Fdo->SessionLock);
 }
 
-NTSTATUS
-WmiRegister(
-    IN  PXENIFACE_FDO   Fdo
+static VOID
+SessionsSuspendLocked(
+    IN  PXENIFACE_FDO       Fdo,
+    IN  PXENSTORE_SESSION   Session
     )
 {
-    NTSTATUS            status;
+    PLIST_ENTRY             ListEntry;
+    PXENSTORE_WATCH         Watch;
 
-    if (Fdo->WmiReady)
-        return STATUS_SUCCESS;
+    ASSERT3P(Fdo->SessionLock.Owner, ==, KeGetCurrentThread());
 
-    Trace("%s\n",__FUNCTION__);
-    Info("DRV: XenIface WMI Initialisation\n");
+    AcquireMutex(&Session->WatchMapLock);
+    for (ListEntry = Session->WatchList.Flink;
+         ListEntry != &Session->WatchList;
+         ListEntry = ListEntry->Flink) {
+        Watch = CONTAINING_RECORD(ListEntry, XENSTORE_WATCH, ListEntry);
 
-    status = IoWMIRegistrationControl(Fdo->Dx->DeviceObject,
-                                      WMIREG_ACTION_REGISTER);
-    if (!NT_SUCCESS(status))
-        goto fail1;
-
-    Fdo->WmiReady = 1;
-    return STATUS_SUCCESS;
+        if (Watch->WatchHandle != NULL)
+            XENBUS_STORE(WatchRemove, &Watch->Fdo->StoreInterface, Watch->WatchHandle);
+        Watch->WatchHandle = NULL;
+    }
+    Session->Suspended = TRUE;
+    ReleaseMutex(&Session->WatchMapLock);
 
-fail1:
-    Error("fail1 (%08x)\n", status);
-    return status;
+    if (Session->Transaction != NULL)
+        XENBUS_STORE(TransactionEnd, &Fdo->StoreInterface, Session->Transaction, FALSE);
+    Session->Transaction = NULL;
 }
 
-VOID
-WmiDeregister(
-    IN  PXENIFACE_FDO   Fdo
+static VOID
+SessionResumeLocked(
+    IN  PXENSTORE_SESSION   Session
     )
 {
-    if (!Fdo->WmiReady)
-        return;
-
-    Info("DRV: XenIface WMI Finalisation\n");
-    Trace("%s\n",__FUNCTION__);
+    PLIST_ENTRY             ListEntry;
+    PXENSTORE_WATCH         Watch;
 
-    SessionsRemoveAll(Fdo);
-    (VOID) IoWMIRegistrationControl(Fdo->Dx->DeviceObject,
-                                    WMIREG_ACTION_DEREGISTER);
-    Fdo->WmiReady = 0;
-}
+    // ASSERT3P(Fdo->SessionLock.Owner, ==, KeGetCurrentThread());
 
-NTSTATUS
-WmiChangeSingleInstance(
-    PXENIFACE_FDO Fdo,
-    PIO_STACK_LOCATION stack
-   )
-{
-    UNREFERENCED_PARAMETER(Fdo);
-    UNREFERENCED_PARAMETER(stack);
-    Trace("%s\n",__FUNCTION__);
-    return STATUS_NOT_SUPPORTED;
-}
+    AcquireMutex(&Session->WatchMapLock);
+    for (ListEntry = Session->WatchList.Flink;
+         ListEntry != &Session->WatchList;
+         ListEntry = ListEntry->Flink) {
+        Watch = CONTAINING_RECORD(ListEntry, XENSTORE_WATCH, ListEntry);
 
-NTSTATUS
-WmiChangeSingleItem(
-    IN PXENIFACE_FDO Fdo,
-    IN PIO_STACK_LOCATION stack
-   )
-{
-    UNREFERENCED_PARAMETER(Fdo);
-    UNREFERENCED_PARAMETER(stack);
-    Trace("%s\n",__FUNCTION__);
-    return STATUS_NOT_SUPPORTED;
-}
+        if (Watch->Finished)
+            continue;
 
-NTSTATUS
-WmiDisableCollection(
-    IN PXENIFACE_FDO Fdo,
-    IN PIO_STACK_LOCATION stack
-   )
-{
-    UNREFERENCED_PARAMETER(Fdo);
-    UNREFERENCED_PARAMETER(stack);
-    Trace("%s\n",__FUNCTION__);
-    return STATUS_NOT_SUPPORTED;
-}
-
-NTSTATUS
-WmiDisableEvents(
-    IN PXENIFACE_FDO Fdo,
-    IN PIO_STACK_LOCATION stack
-   )
-{
-    UNREFERENCED_PARAMETER(Fdo);
-    UNREFERENCED_PARAMETER(stack);
-    Trace("%s\n",__FUNCTION__);
-    return STATUS_NOT_SUPPORTED;
-}
+        Watch->SuspendCount = XENBUS_SUSPEND(GetCount, &Watch->Fdo->SuspendInterface);
+        StartWatch(Watch->Fdo, Watch);
+    }
+    Session->Suspended = FALSE;
 
-NTSTATUS
-WmiEnableCollection(
-    IN PXENIFACE_FDO Fdo,
-    IN PIO_STACK_LOCATION stack
-   )
-{
-    UNREFERENCED_PARAMETER(Fdo);
-    UNREFERENCED_PARAMETER(stack);
-    Trace("%s\n",__FUNCTION__);
-    return STATUS_NOT_SUPPORTED;
+    Session->Changed = TRUE;
+    KeSetEvent(&Session->SessionChangedEvent, IO_NO_INCREMENT,FALSE);
+    ReleaseMutex(&Session->WatchMapLock);
 }
 
-NTSTATUS
-WmiEnableEvents(
-    IN PXENIFACE_FDO Fdo,
-    IN PIO_STACK_LOCATION stack
-   )
+static NTSTATUS
+NodeTooSmall(
+    IN  UCHAR*      Buffer,
+    IN  ULONG       BufferSize,
+    IN  ULONG       Needed,
+    OUT ULONG_PTR*  BytesWritten
+    )
 {
-    UNREFERENCED_PARAMETER(Fdo);
-    UNREFERENCED_PARAMETER(stack);
-    Trace("%s\n",__FUNCTION__);
-    return STATUS_NOT_SUPPORTED;
-}
+    WNODE_TOO_SMALL*    node;
+    ULONG               RequiredSize;
 
-NTSTATUS NodeTooSmall(UCHAR *Buffer,
-                    ULONG BufferSize,
-                    ULONG Needed,
-                    ULONG_PTR *byteswritten) {
-    WNODE_TOO_SMALL *node;
-    ULONG RequiredSize;
     if (!AccessWmiBuffer(Buffer, FALSE, &RequiredSize, BufferSize,
-                            WMI_BUFFER, sizeof(WNODE_TOO_SMALL), &node,
-                            WMI_DONE))
-    {
-        *byteswritten = RequiredSize;
+                         WMI_BUFFER, sizeof(WNODE_TOO_SMALL), &node,
+                         WMI_DONE)) {
+        *BytesWritten = RequiredSize;
         return STATUS_BUFFER_TOO_SMALL;
     }
-    node->WnodeHeader.BufferSize=sizeof(WNODE_TOO_SMALL);
+
+    node->WnodeHeader.BufferSize = sizeof(WNODE_TOO_SMALL);
     KeQuerySystemTime(&node->WnodeHeader.TimeStamp);
     node->WnodeHeader.Flags = WNODE_FLAG_TOO_SMALL;
     node->SizeNeeded = Needed;
-    *byteswritten = sizeof(WNODE_TOO_SMALL);
+
+    *BytesWritten = sizeof(WNODE_TOO_SMALL);
     return STATUS_SUCCESS;
 }
 
-NTSTATUS
-SessionExecuteRemoveValue(UCHAR *InBuffer,
-                            ULONG InBufferSize,
-                            UCHAR *OutBuffer,
-                            ULONG OutBufferSize,
-                            XENIFACE_FDO* fdoData,
-                            UNICODE_STRING *instance,
-                            OUT ULONG_PTR *byteswritten) {
-    ULONG RequiredSize;
-    NTSTATUS status;
-    UCHAR* upathname;
-    OEM_STRING pathname;
-    XenStoreSession *session;
-    char *tmpbuffer;
-
-    *byteswritten=0;
+static NTSTATUS
+SessionExecuteRemoveValue(
+    IN  PXENIFACE_FDO   Fdo,
+    IN  PUNICODE_STRING instance,
+    IN  UCHAR*          InBuffer,
+    IN  ULONG           InBufferSize,
+    IN  UCHAR*          OutBuffer,
+    IN  ULONG           OutBufferSize,
+    OUT ULONG_PTR*      BytesWritten
+    )
+{
+    ULONG               RequiredSize;
+    NTSTATUS            status;
+    UCHAR*              upathname;
+    OEM_STRING          pathname;
+    PXENSTORE_SESSION   session;
+    char*               tmpbuffer;
+
+    *BytesWritten = 0;
+    status = STATUS_INVALID_DEVICE_REQUEST;
     if (!AccessWmiBuffer(InBuffer, TRUE, &RequiredSize, InBufferSize,
-                            WMI_STRING, &upathname,
-                            WMI_DONE))
-        return STATUS_INVALID_DEVICE_REQUEST;
-    if (!fdoData->InterfacesAcquired) {
-        return STATUS_INSUFFICIENT_RESOURCES;
-    }
+                         WMI_STRING, &upathname,
+                         WMI_DONE))
+        goto fail1;
+
+    status = STATUS_INSUFFICIENT_RESOURCES;
+    if (!Fdo->InterfacesAcquired)
+        goto fail2;
 
     status = GetCountedUTF8String(&pathname, upathname);
     if (!NT_SUCCESS(status))
-        return status;
+        goto fail3;
 
     status = STATUS_INSUFFICIENT_RESOURCES;
     tmpbuffer = WmiAllocate(pathname.Length + 1);
-    if (!tmpbuffer) {
-        goto fail1;
-    }
+    if (tmpbuffer == NULL)
+        goto fail4;
 
     RtlCopyBytes(tmpbuffer, pathname.Buffer, pathname.Length);
 
     status = STATUS_WMI_INSTANCE_NOT_FOUND;
-    if ((session = FindSessionByInstanceAndLock(fdoData, instance)) ==
-            NULL){
-        goto fail2;
-    }
-    status = XENBUS_STORE(Remove, &fdoData->StoreInterface, session->transaction, NULL, tmpbuffer);
-    UnlockSessions(fdoData);
+    session = FindSessionByInstanceAndLock(Fdo, instance);
+    if (session == NULL)
+        goto fail5;
+
+    status = XENBUS_STORE(Remove, &Fdo->StoreInterface, session->Transaction, NULL, tmpbuffer);
+    ReleaseMutex(&Fdo->SessionLock);
+
+    if (!NT_SUCCESS(status))
+        goto fail6;
 
-fail2:
     WmiFree(tmpbuffer);
+    FreeUTF8String(&pathname);
 
-fail1:
+    return STATUS_SUCCESS;
+
+fail6:
+fail5:
+    WmiFree(tmpbuffer);
+fail4:
     FreeUTF8String(&pathname);
+fail3:
+fail2:
+fail1:
     return status;
-
 }
 
-NTSTATUS
-SessionExecuteRemoveWatch(UCHAR *InBuffer,
-                            ULONG InBufferSize,
-                            UCHAR *OutBuffer,
-                            ULONG OutBufferSize,
-                            XENIFACE_FDO* fdoData,
-                            UNICODE_STRING *instance,
-                            OUT ULONG_PTR *byteswritten) {
-    ULONG RequiredSize;
-    UCHAR* upathname;
-    XenStoreWatch* watch;
-    UNICODE_STRING unicpath_notbacked;
-    XenStoreSession *session;
-
+static NTSTATUS
+SessionExecuteRemoveWatch(
+    IN  PXENIFACE_FDO   Fdo,
+    IN  PUNICODE_STRING instance,
+    IN  UCHAR*          InBuffer,
+    IN  ULONG           InBufferSize,
+    IN  UCHAR*          OutBuffer,
+    IN  ULONG           OutBufferSize,
+    OUT ULONG_PTR*      BytesWritten
+    )
+{
+    NTSTATUS            status;
+    ULONG               RequiredSize;
+    UCHAR*              upathname;
+    PXENSTORE_WATCH     watch;
+    UNICODE_STRING      unicpath_notbacked;
+    PXENSTORE_SESSION   session;
+
+    *BytesWritten = 0;
+    status = STATUS_INVALID_DEVICE_REQUEST;
     if (!AccessWmiBuffer(InBuffer, TRUE, &RequiredSize, InBufferSize,
-                            WMI_STRING, &upathname,
-                            WMI_DONE))
-        return STATUS_INVALID_DEVICE_REQUEST;
-
+                         WMI_STRING, &upathname,
+                         WMI_DONE))
+        goto fail1;
 
     GetCountedUnicodeString(&unicpath_notbacked, upathname);
 
-    if ((session = FindSessionByInstanceAndLock(fdoData, instance)) ==
-            NULL){
-        return STATUS_WMI_INSTANCE_NOT_FOUND;
-    }
-
-
-    Trace("Find Watch\n");
+    status = STATUS_WMI_INSTANCE_NOT_FOUND;
+    session = FindSessionByInstanceAndLock(Fdo, instance);
+    if (session == NULL)
+        goto fail2;
 
+    AcquireMutex(&session->WatchMapLock);
     watch = SessionFindWatchLocked(session, &unicpath_notbacked);
-
     if (watch) {
-
-        SessionRemoveWatchLocked(session, watch);
-    }
-    else {
+        SessionRemoveWatchLocked(watch);
+    } else {
         Warning("No Watch\n");
     }
-#pragma prefast (suppress:26110)
     ReleaseMutex(&session->WatchMapLock);
-    UnlockSessions(fdoData);
-
-    *byteswritten=0;
-
 
+    ReleaseMutex(&Fdo->SessionLock);
 
     return STATUS_SUCCESS;
 
+fail2:
+fail1:
+    return status;
 }
 
+static NTSTATUS
+SessionExecuteSetWatch(
+    IN  PXENIFACE_FDO   Fdo,
+    IN  PUNICODE_STRING instance,
+    IN  UCHAR*          InBuffer,
+    IN  ULONG           InBufferSize,
+    IN  UCHAR*          OutBuffer,
+    IN  ULONG           OutBufferSize,
+    OUT ULONG_PTR*      BytesWritten
+    )
+{
+    ULONG               RequiredSize;
+    NTSTATUS            status;
+    UCHAR*              upathname;
+    PXENSTORE_SESSION   Session;
+    UNICODE_STRING      unicpath_notbacked;
+    UNICODE_STRING      unicpath_backed;
 
-NTSTATUS
-SessionExecuteSetWatch(UCHAR *InBuffer,
-                            ULONG InBufferSize,
-                            UCHAR *OutBuffer,
-                            ULONG OutBufferSize,
-                            XENIFACE_FDO* fdoData,
-                            UNICODE_STRING *instance,
-                            OUT ULONG_PTR *byteswritten) {
-    ULONG RequiredSize;
-    NTSTATUS status;
-    UCHAR* upathname;
-    XenStoreWatch* watch;
-    XenStoreSession *session;
-    UNICODE_STRING unicpath_notbacked;
-    UNICODE_STRING unicpath_backed;
+    *BytesWritten = 0;
+    status = STATUS_INVALID_DEVICE_REQUEST;
     if (!AccessWmiBuffer(InBuffer, TRUE, &RequiredSize, InBufferSize,
-                            WMI_STRING, &upathname,
-                            WMI_DONE))
-        return STATUS_INVALID_DEVICE_REQUEST;
-
+                         WMI_STRING, &upathname,
+                         WMI_DONE))
+        goto fail1;
 
     GetCountedUnicodeString(&unicpath_notbacked, upathname);
-    status = CloneUnicodeString(&unicpath_backed, &unicpath_notbacked);
-    if (!NT_SUCCESS(status)) return status;
-
-    if ((session = FindSessionByInstanceAndLock(fdoData, instance)) ==
-            NULL){
-        FreeUnicodeStringBuffer(&unicpath_backed);
-        return STATUS_WMI_INSTANCE_NOT_FOUND;
-    }
 
-    status = SessionAddWatchLocked(session, fdoData, &unicpath_backed, &watch);
+    status = CloneUnicodeString(&unicpath_backed, &unicpath_notbacked);
+    if (!NT_SUCCESS(status))
+        goto fail2;
 
-    UnlockSessions(fdoData);
-    if (!NT_SUCCESS(status)) {
-        FreeUnicodeStringBuffer(&unicpath_backed);
-        return status;
-    }
+    status = STATUS_WMI_INSTANCE_NOT_FOUND;
+    Session = FindSessionByInstanceAndLock(Fdo, instance);
+    if (Session == NULL)
+        goto fail3;
 
+    AcquireMutex(&Session->WatchMapLock);
 
-    *byteswritten=0;
+    status = SessionAddWatchLocked(Fdo, Session, &unicpath_backed);
 
+    ReleaseMutex(&Session->WatchMapLock);
 
+    ReleaseMutex(&Fdo->SessionLock);
+    if (!NT_SUCCESS(status))
+        goto fail4;
 
     return STATUS_SUCCESS;
 
+fail4:
+fail3:
+    FreeUnicodeStringBuffer(&unicpath_backed);
+fail2:
+fail1:
+    return status;
 }
-NTSTATUS
-SessionExecuteEndSession(UCHAR *InBuffer,
-                            ULONG InBufferSize,
-                            UCHAR *OutBuffer,
-                            ULONG OutBufferSize,
-                            XENIFACE_FDO* fdoData,
-                            UNICODE_STRING *instance,
-                            OUT ULONG_PTR *byteswritten) {
-    XenStoreSession *session;
-    Trace("ExecuteEndSession\n");
-    *byteswritten = 0;
-    if ((session = FindSessionByInstanceAndLock(fdoData, instance)) ==
-            NULL){
-        return STATUS_WMI_INSTANCE_NOT_FOUND;
-    }
 
-    RemoveSessionLocked(fdoData, session);
-    UnlockSessions(fdoData);
+static NTSTATUS
+SessionExecuteEndSession(
+    IN  PXENIFACE_FDO   Fdo,
+    IN  PUNICODE_STRING instance,
+    IN  UCHAR*          InBuffer,
+    IN  ULONG           InBufferSize,
+    IN  UCHAR*          OutBuffer,
+    IN  ULONG           OutBufferSize,
+    OUT ULONG_PTR*      BytesWritten
+    )
+{
+    PXENSTORE_SESSION   Session;
+    NTSTATUS            status;
+
+    *BytesWritten = 0;
+    status = STATUS_WMI_INSTANCE_NOT_FOUND;
+    Session = FindSessionByInstanceAndLock(Fdo, instance);
+    if (Session == NULL)
+        goto fail1;
+
+    SessionRemoveLocked(Fdo, Session);
+    ReleaseMutex(&Fdo->SessionLock);
+
     return STATUS_SUCCESS;
+
+fail1:
+    return status;
 }
-NTSTATUS
-SessionExecuteSetValue(UCHAR *InBuffer,
-                            ULONG InBufferSize,
-                            UCHAR *OutBuffer,
-                            ULONG OutBufferSize,
-                            XENIFACE_FDO* fdoData,
-                            UNICODE_STRING *instance,
-                            OUT ULONG_PTR *byteswritten) {
-    ULONG RequiredSize;
-    NTSTATUS status;
-    UCHAR* upathname;
-    UCHAR* uvalue;
-    OEM_STRING pathname;
-    OEM_STRING value;
-    XenStoreSession *session;
-    char *tmppath;
-    char* tmpvalue;
-
-    Trace(" Try to write\n");
+
+static NTSTATUS
+SessionExecuteSetValue(
+    IN  PXENIFACE_FDO       Fdo,
+    IN  PUNICODE_STRING     instance,
+    IN  UCHAR*              InBuffer,
+    IN  ULONG               InBufferSize,
+    IN  UCHAR*              OutBuffer,
+    IN  ULONG               OutBufferSize,
+    OUT ULONG_PTR*          BytesWritten
+    )
+{
+    ULONG                   RequiredSize;
+    NTSTATUS                status;
+    UCHAR*                  upathname;
+    UCHAR*                  uvalue;
+    OEM_STRING              pathname;
+    OEM_STRING              value;
+    PXENSTORE_SESSION       session;
+    char*                   tmppath;
+    char*                   tmpvalue;
+
+    *BytesWritten = 0;
+    status = STATUS_INVALID_DEVICE_REQUEST;
     if (!AccessWmiBuffer(InBuffer, TRUE, &RequiredSize, InBufferSize,
-                            WMI_STRING, &upathname,
-                            WMI_STRING, &uvalue,
-                            WMI_DONE))
-        return STATUS_INVALID_DEVICE_REQUEST;
-    if (!fdoData->InterfacesAcquired) {
-        return STATUS_INSUFFICIENT_RESOURCES;
-    }
+                         WMI_STRING, &upathname,
+                         WMI_STRING, &uvalue,
+                         WMI_DONE))
+        goto fail1;
+
+    status = STATUS_INSUFFICIENT_RESOURCES;
+    if (!Fdo->InterfacesAcquired)
+        goto fail2;
+
     status = GetCountedUTF8String(&pathname, upathname);
     if (!NT_SUCCESS(status))
-        return status;
+        goto fail3;
 
     status = STATUS_INSUFFICIENT_RESOURCES;
     tmppath = WmiAllocate(pathname.Length + 1);
-    if (!tmppath) {
-        goto fail1;
-    }
+    if (tmppath == NULL)
+        goto fail4;
 
     RtlCopyBytes(tmppath, pathname.Buffer, pathname.Length);
 
     status = GetCountedUTF8String(&value, uvalue);
-    if (!NT_SUCCESS(status)){
-        goto fail2;
-    }
+    if (!NT_SUCCESS(status))
+        goto fail5;
+
     status = STATUS_INSUFFICIENT_RESOURCES;
     tmpvalue = WmiAllocate(value.Length + 1);
-    if (!tmpvalue) {
-        goto fail3;
-    }
+    if (tmpvalue == NULL)
+        goto fail6;
 
     RtlCopyBytes(tmpvalue, value.Buffer, value.Length);
 
     status = STATUS_WMI_INSTANCE_NOT_FOUND;
-    if ((session = FindSessionByInstanceAndLock(fdoData, instance)) ==
-            NULL){
-        goto fail4;
-    }
-    status = XENBUS_STORE(Printf, &fdoData->StoreInterface, session->transaction, NULL, tmppath, "%s", tmpvalue);
-    Trace(" Write %s to %s (%p)\n", tmpvalue, tmppath, status);
-    UnlockSessions(fdoData);
+    session = FindSessionByInstanceAndLock(Fdo, instance);
+    if (session == NULL)
+        goto fail7;
 
-fail4:
-    WmiFree(tmpvalue);
+    status = XENBUS_STORE(Printf, &Fdo->StoreInterface, session->Transaction, NULL, tmppath, "%s", tmpvalue);
+    ReleaseMutex(&Fdo->SessionLock);
 
-fail3:
-    FreeUTF8String(&value);
+    if (!NT_SUCCESS(status))
+        goto fail8;
 
-fail2:
+    WmiFree(tmpvalue);
+    FreeUTF8String(&value);
     WmiFree(tmppath);
-
-fail1:
     FreeUTF8String(&pathname);
 
-    *byteswritten = 0;
-    return status;
+    return STATUS_SUCCESS;
 
+fail8:
+fail7:
+    WmiFree(tmpvalue);
+fail6:
+    FreeUTF8String(&value);
+fail5:
+    WmiFree(tmppath);
+fail4:
+    FreeUTF8String(&pathname);
+fail3:
+fail2:
+fail1:
+    return status;
 }
-NTSTATUS
-SessionExecuteGetFirstChild(UCHAR *InBuffer,
-                            ULONG InBufferSize,
-                            UCHAR *OutBuffer,
-                            ULONG OutBufferSize,
-                            XENIFACE_FDO* fdoData,
-                            UNICODE_STRING *instance,
-                            OUT ULONG_PTR *byteswritten) {
-    ULONG RequiredSize;
-    UCHAR *uloc;
-    NTSTATUS status;
-    OEM_STRING path;
-    PCHAR listresults;
-    size_t stringarraysize;
-    UCHAR *valuepos;
-    XenStoreSession *session;
-    char *tmppath;
+
+static NTSTATUS
+SessionExecuteGetFirstChild(
+    IN  PXENIFACE_FDO       Fdo,
+    IN  PUNICODE_STRING     instance,
+    IN  UCHAR*              InBuffer,
+    IN  ULONG               InBufferSize,
+    IN  UCHAR*              OutBuffer,
+    IN  ULONG               OutBufferSize,
+    OUT ULONG_PTR*          BytesWritten
+    )
+{
+    ULONG                   RequiredSize;
+    UCHAR*                  uloc;
+    NTSTATUS                status;
+    OEM_STRING              path;
+    PCHAR                   listresults;
+    size_t                  stringarraysize;
+    UCHAR*                  valuepos;
+    PXENSTORE_SESSION       session;
+    char*                   tmppath;
+
+    *BytesWritten = 0;
+    status = STATUS_INVALID_DEVICE_REQUEST;
     if (!AccessWmiBuffer(InBuffer, TRUE, &RequiredSize, InBufferSize,
-                            WMI_STRING, &uloc,
-                            WMI_DONE)){
-        return  STATUS_INVALID_DEVICE_REQUEST;
-    }
-    if (!fdoData->InterfacesAcquired) {
-        return STATUS_INSUFFICIENT_RESOURCES;
-    }
+                         WMI_STRING, &uloc,
+                         WMI_DONE))
+        goto fail1;
 
-    status = GetCountedUTF8String(&path, uloc);
+    status = STATUS_INSUFFICIENT_RESOURCES;
+    if (!Fdo->InterfacesAcquired)
+        goto fail2;
 
-    if (!NT_SUCCESS(status)) {
-        return status;
-    }
+    status = GetCountedUTF8String(&path, uloc);
+    if (!NT_SUCCESS(status))
+        goto fail3;
 
     status = STATUS_INSUFFICIENT_RESOURCES;
     tmppath = WmiAllocate(path.Length + 1);
-    if (!tmppath) {
-        goto fail1;
-    }
+    if (tmppath == NULL)
+        goto fail4;
 
     RtlCopyBytes(tmppath, path.Buffer, path.Length);
 
     status = STATUS_WMI_INSTANCE_NOT_FOUND;
-    if ((session = FindSessionByInstanceAndLock(fdoData, instance)) ==
-            NULL){
-        goto fail2;
-    }
-    status = XENBUS_STORE(Directory,&fdoData->StoreInterface, session->transaction, NULL, tmppath, &listresults);
-    UnlockSessions(fdoData);
+    session = FindSessionByInstanceAndLock(Fdo, instance);
+    if (session == NULL)
+        goto fail5;
 
-    if (!NT_SUCCESS(status)) {
-        goto fail2;
-    }
+    status = XENBUS_STORE(Directory, &Fdo->StoreInterface, session->Transaction, NULL, tmppath, &listresults);
+    ReleaseMutex(&Fdo->SessionLock);
+
+    if (!NT_SUCCESS(status))
+        goto fail6;
 
     stringarraysize = 0;
     if ((listresults != NULL) && (listresults[0] != 0)) {
@@ -1822,108 +1733,110 @@ SessionExecuteGetFirstChild(UCHAR *InBuffer,
         }
         stringarraysize += GetCountedUtf8Size(listresults);
     } else {
-        stringarraysize+=GetCountedUtf8Size("");
+        stringarraysize += GetCountedUtf8Size("");
     }
 
     status = STATUS_BUFFER_TOO_SMALL;
     if (!AccessWmiBuffer(InBuffer, FALSE, &RequiredSize, OutBufferSize,
-                            WMI_STRING, stringarraysize, &valuepos,
-                            WMI_DONE)){
-        goto fail3;
-    }
+                         WMI_STRING, stringarraysize, &valuepos,
+                         WMI_DONE))
+        goto fail7;
 
-    status = STATUS_SUCCESS;
     if ((listresults != NULL) && (listresults[0] != 0)) {
         PSTR fullpath;
-        if ((path.Length == 1) && (path.Buffer[0] == '/')) {
+        if ((path.Length == 1) && (path.Buffer[0] == '/'))
             fullpath = Xmasprintf("/%s", listresults);
-        } else {
+        else
             fullpath = Xmasprintf("%s/%s", path.Buffer, listresults);
-        }
 
-        if (fullpath == NULL) {
-            status = STATUS_NO_MEMORY;
-            goto fail4;
-        }
+        status = STATUS_NO_MEMORY;
+        if (fullpath == NULL)
+            goto fail8;
 
         WriteCountedUTF8String(fullpath, valuepos);
-        valuepos+=GetCountedUtf8Size(fullpath);
+        valuepos += GetCountedUtf8Size(fullpath);
         WmiFree(fullpath);
-    }
-    else {
+    } else {
         WriteCountedUTF8String("", valuepos);
     }
 
-fail4:
-fail3:
-    XENBUS_STORE(Free, &fdoData->StoreInterface, listresults);
-
-    *byteswritten = RequiredSize;
+    XENBUS_STORE(Free, &Fdo->StoreInterface, listresults);
 
-fail2:
     WmiFree(tmppath);
-
-fail1:
     FreeUTF8String(&path);
 
-    return status;
+    *BytesWritten = RequiredSize;
+    return STATUS_SUCCESS;
 
+fail8:
+fail7:
+    XENBUS_STORE(Free, &Fdo->StoreInterface, listresults);
+    *BytesWritten = RequiredSize;
+fail6:
+fail5:
+    WmiFree(tmppath);
+fail4:
+    FreeUTF8String(&path);
+fail3:
+fail2:
+fail1:
+    return status;
 }
 
-NTSTATUS
-SessionExecuteGetNextSibling(UCHAR *InBuffer,
-                            ULONG InBufferSize,
-                            UCHAR *OutBuffer,
-                            ULONG OutBufferSize,
-                            XENIFACE_FDO* fdoData,
-                            UNICODE_STRING *instance,
-                            OUT ULONG_PTR *byteswritten) {
-    ULONG RequiredSize;
-    UCHAR *uloc;
-    NTSTATUS status;
-    OEM_STRING path;
-    ANSI_STRING checkleaf;
-    PCHAR listresults;
-    PCHAR nextresult;
-    size_t stringarraysize;
-    UCHAR *valuepos;
-    XenStoreSession *session;
-    char *tmppath;
-    char *tmpleaf;
-    int leafoffset;
-    char *attemptstring;
+static NTSTATUS
+SessionExecuteGetNextSibling(
+    IN  PXENIFACE_FDO       Fdo,
+    IN  PUNICODE_STRING     instance,
+    IN  UCHAR*              InBuffer,
+    IN  ULONG               InBufferSize,
+    IN  UCHAR*              OutBuffer,
+    IN  ULONG               OutBufferSize,
+    OUT ULONG_PTR*          BytesWritten
+    )
+{
+    ULONG                   RequiredSize;
+    UCHAR*                  uloc;
+    NTSTATUS                status;
+    OEM_STRING              path;
+    ANSI_STRING             checkleaf;
+    PCHAR                   listresults;
+    PCHAR                   nextresult;
+    size_t                  stringarraysize;
+    UCHAR*                  valuepos;
+    PXENSTORE_SESSION       session;
+    char*                   tmppath;
+    char*                   tmpleaf;
+    int                     leafoffset;
+    char*                   attemptstring;
+
+    *BytesWritten = 0;
+    status = STATUS_INVALID_DEVICE_REQUEST;
     if (!AccessWmiBuffer(InBuffer, TRUE, &RequiredSize, InBufferSize,
-                            WMI_STRING, &uloc,
-                            WMI_DONE)){
-        return  STATUS_INVALID_DEVICE_REQUEST;
-    }
-    if (!fdoData->InterfacesAcquired) {
-        return STATUS_INSUFFICIENT_RESOURCES;
-    }
+                         WMI_STRING, &uloc,
+                         WMI_DONE))
+        goto fail1;
 
-    status = GetCountedUTF8String(&path, uloc);
+    status = STATUS_INSUFFICIENT_RESOURCES;
+    if (!Fdo->InterfacesAcquired)
+        goto fail2;
 
-    if (!NT_SUCCESS(status)) {
-        return status;
-    }
+    status = GetCountedUTF8String(&path, uloc);
+    if (!NT_SUCCESS(status))
+        goto fail3;
 
     status = STATUS_INSUFFICIENT_RESOURCES;
     tmppath = WmiAllocate(path.Length + 1);
-
-    if (!tmppath) {
-        goto fail1;
-    }
+    if (tmppath == NULL)
+        goto fail4;
 
     tmpleaf = WmiAllocate(path.Length + 1);
-    if (!tmpleaf) {
-        goto fail2;
-    }
+    if (tmpleaf == NULL)
+        goto fail5;
 
     status = STATUS_WMI_INSTANCE_NOT_FOUND;
-    if ((session = FindSessionByInstanceAndLock(fdoData, instance)) ==
-            NULL){
-        goto fail3;
-    }
+    session = FindSessionByInstanceAndLock(Fdo, instance);
+    if (session == NULL)
+        goto fail6;
 
     leafoffset = 0;
     if (path.Length > 1) {
@@ -1936,7 +1849,7 @@ SessionExecuteGetNextSibling(UCHAR *InBuffer,
         RtlCopyBytes(tmppath, path.Buffer, leafoffset);
         RtlCopyBytes(tmpleaf, path.Buffer + leafoffset + 1, path.Length - leafoffset - 1);
     } else if (path.Buffer[0] == '/') {
-        if (path.Length>1)
+        if (path.Length > 1)
             RtlCopyBytes(tmpleaf, path.Buffer + 1, path.Length - 1);
         tmppath[0] = '/';
     } else {
@@ -1944,39 +1857,34 @@ SessionExecuteGetNextSibling(UCHAR *InBuffer,
         RtlCopyBytes(tmpleaf, path.Buffer, path.Length);
     }
 
-    status = XENBUS_STORE(Directory,&fdoData->StoreInterface, session->transaction, NULL, tmppath, &listresults);
-    UnlockSessions(fdoData);
+    status = XENBUS_STORE(Directory, &Fdo->StoreInterface, session->Transaction, NULL, tmppath, &listresults);
+    ReleaseMutex(&Fdo->SessionLock);
 
-    if (!NT_SUCCESS(status)) {
-        goto fail3;
-    }
+    if (!NT_SUCCESS(status))
+        goto fail7;
 
     stringarraysize = 0;
     RtlInitAnsiString(&checkleaf, tmpleaf);
 
     nextresult = listresults;
-
     while (*nextresult != 0) {
         ANSI_STRING checkstr;
         RtlInitAnsiString(&checkstr, nextresult);
-        if (RtlEqualString(&checkstr, &checkleaf, TRUE)) {
+        if (RtlEqualString(&checkstr, &checkleaf, TRUE))
             break;
-        }
-        while (*nextresult!=0) {
+
+        while (*nextresult != 0)
             nextresult++;
-        }
         nextresult++;
     }
 
-
     attemptstring = NULL;
-    while (*nextresult !=0) {
+    while (*nextresult != 0)
         nextresult++;
-    }
     nextresult++;
-    if (*nextresult!=0) {
+
+    if (*nextresult != 0)
         attemptstring = nextresult;
-    }
 
     if (attemptstring != NULL) {
         stringarraysize += CountBytesUtf16FromUtf8(tmppath); //sizeof(WCHAR)*leafoffset;
@@ -1992,105 +1900,109 @@ SessionExecuteGetNextSibling(UCHAR *InBuffer,
 
     status = STATUS_BUFFER_TOO_SMALL;
     if (!AccessWmiBuffer(InBuffer, FALSE, &RequiredSize, OutBufferSize,
-                            WMI_STRING, stringarraysize, &valuepos,
-                            WMI_DONE)){
-        goto fail4;
-    }
+                         WMI_STRING, stringarraysize, &valuepos,
+                         WMI_DONE))
+        goto fail8;
 
-    status = STATUS_SUCCESS;
     if (attemptstring != NULL) {
         PSTR fullpath;
-        if ((leafoffset == 1) && (path.Buffer[0] == '/')) {
+        if ((leafoffset == 1) && (path.Buffer[0] == '/'))
             fullpath = Xmasprintf("/%s", attemptstring);
-        } else {
+        else
             fullpath = Xmasprintf("%s/%s", tmppath, attemptstring);
-        }
 
-        if (fullpath == NULL) {
-            status = STATUS_NO_MEMORY;
-            goto fail5;
-        }
+        status = STATUS_NO_MEMORY;
+        if (fullpath == NULL)
+            goto fail9;
 
         WriteCountedUTF8String(fullpath, valuepos);
         WmiFree(fullpath);
-    }
-    else {
+    } else {
         WriteCountedUTF8String("", valuepos);
-        valuepos+=GetCountedUtf8Size("");
+        valuepos += GetCountedUtf8Size("");
     }
 
+    XENBUS_STORE(Free, &Fdo->StoreInterface, listresults);
+
+    WmiFree(tmpleaf);
+    WmiFree(tmppath);
+    FreeUTF8String(&path);
+
+    *BytesWritten = RequiredSize;
+    return STATUS_SUCCESS;
 
+fail9:
+fail8:
+    XENBUS_STORE(Free, &Fdo->StoreInterface, listresults);
+    *BytesWritten = RequiredSize;
+fail7:
+fail6:
+    WmiFree(tmpleaf);
 fail5:
+    WmiFree(tmppath);
 fail4:
-    XENBUS_STORE(Free, &fdoData->StoreInterface, listresults);
-
+    FreeUTF8String(&path);
 fail3:
-    WmiFree(tmpleaf);
-
 fail2:
-    WmiFree(tmppath);
-
 fail1:
-    FreeUTF8String(&path);
-    *byteswritten = RequiredSize;
     return status;
-
 }
 
-NTSTATUS
-SessionExecuteGetChildren(UCHAR *InBuffer,
-                            ULONG InBufferSize,
-                            UCHAR *OutBuffer,
-                            ULONG OutBufferSize,
-                            XENIFACE_FDO* fdoData,
-                            UNICODE_STRING *instance,
-                            OUT ULONG_PTR *byteswritten) {
-    int i;
-    ULONG RequiredSize;
-    UCHAR *uloc;
-    NTSTATUS status;
-    OEM_STRING path;
-    PCHAR listresults;
-    PCHAR nextresults;
-    ULONG *noofnodes;
-    size_t stringarraysize;
-    UCHAR *valuepos;
-    XenStoreSession *session;
-    char *tmppath;
+static NTSTATUS
+SessionExecuteGetChildren(
+    IN  PXENIFACE_FDO       Fdo,
+    IN  PUNICODE_STRING     instance,
+    IN  UCHAR*              InBuffer,
+    IN  ULONG               InBufferSize,
+    IN  UCHAR*              OutBuffer,
+    IN  ULONG               OutBufferSize,
+    OUT ULONG_PTR*          BytesWritten
+    )
+{
+    int                     i;
+    ULONG                   RequiredSize;
+    UCHAR*                  uloc;
+    NTSTATUS                status;
+    OEM_STRING              path;
+    PCHAR                   listresults;
+    PCHAR                   nextresults;
+    ULONG*                  noofnodes;
+    size_t                  stringarraysize;
+    UCHAR*                  valuepos;
+    PXENSTORE_SESSION       session;
+    char*                   tmppath;
+
+    status = STATUS_INVALID_DEVICE_REQUEST;
     if (!AccessWmiBuffer(InBuffer, TRUE, &RequiredSize, InBufferSize,
-                            WMI_STRING, &uloc,
-                            WMI_DONE)){
-        return  STATUS_INVALID_DEVICE_REQUEST;
-    }
-    if (!fdoData->InterfacesAcquired) {
-        return STATUS_INSUFFICIENT_RESOURCES;
-    }
+                         WMI_STRING, &uloc,
+                         WMI_DONE))
+        goto fail1;
 
-    status = GetCountedUTF8String(&path, uloc);
+    status = STATUS_INSUFFICIENT_RESOURCES;
+    if (!Fdo->InterfacesAcquired)
+        goto fail2;
 
-    if (!NT_SUCCESS(status)) {
-        return status;
-    }
+    status = GetCountedUTF8String(&path, uloc);
+    if (!NT_SUCCESS(status))
+        goto fail3;
 
     status = STATUS_INSUFFICIENT_RESOURCES;
     tmppath = WmiAllocate(path.Length + 1);
-    if (!tmppath) {
-        goto fail1;
-    }
+    if (tmppath == NULL)
+        goto fail4;
 
     RtlCopyBytes(tmppath, path.Buffer, path.Length);
 
     status = STATUS_WMI_INSTANCE_NOT_FOUND;
-    if ((session = FindSessionByInstanceAndLock(fdoData, instance)) ==
-            NULL){
-        goto fail2;
-    }
-    status = XENBUS_STORE(Directory,&fdoData->StoreInterface,session->transaction,NULL, tmppath, &listresults);
-    UnlockSessions(fdoData);
+    session = FindSessionByInstanceAndLock(Fdo, instance);
+    if (session == NULL)
+        goto fail5;
 
-    if (!NT_SUCCESS(status)) {
-        goto fail2;
-    }
+    status = XENBUS_STORE(Directory, &Fdo->StoreInterface, session->Transaction, NULL, tmppath, &listresults);
+    ReleaseMutex(&Fdo->SessionLock);
+
+    if (!NT_SUCCESS(status))
+        goto fail6;
 
     stringarraysize = 0;
 
@@ -2111,27 +2023,23 @@ SessionExecuteGetChildren(UCHAR *InBuffer,
 
     status = STATUS_BUFFER_TOO_SMALL;
     if (!AccessWmiBuffer(InBuffer, FALSE, &RequiredSize, OutBufferSize,
-                            WMI_UINT32, &noofnodes,
-                            WMI_STRING, stringarraysize, &valuepos,
-                            WMI_DONE)){
-        goto fail3;
-    }
+                         WMI_UINT32, &noofnodes,
+                         WMI_STRING, stringarraysize, &valuepos,
+                         WMI_DONE))
+        goto fail7;
 
-    status = STATUS_SUCCESS;
     nextresults = listresults;
     i = 0;
     while (*nextresults != 0) {
         PSTR fullpath;
-        if ((path.Length == 1) && (path.Buffer[0] == '/')) {
+        if ((path.Length == 1) && (path.Buffer[0] == '/'))
             fullpath = Xmasprintf("/%s", nextresults);
-        } else {
+        else
             fullpath = Xmasprintf("%s/%s", path.Buffer, nextresults);
-        }
 
-        if (fullpath == NULL) {
-            status = STATUS_NO_MEMORY;
-            goto fail4;
-        }
+        status = STATUS_NO_MEMORY;
+        if (fullpath == NULL)
+            goto fail8;
 
         WriteCountedUTF8String(fullpath, valuepos);
         valuepos += GetCountedUtf8Size(fullpath);
@@ -2144,562 +2052,626 @@ SessionExecuteGetChildren(UCHAR *InBuffer,
     }
     *noofnodes = i;
 
+    XENBUS_STORE(Free, &Fdo->StoreInterface, listresults);
+    WmiFree(tmppath);
+    FreeUTF8String(&path);
+
+    *BytesWritten = RequiredSize;
+    return STATUS_SUCCESS;
+
+fail8:
+fail7:
+    XENBUS_STORE(Free, &Fdo->StoreInterface, listresults);
+    *BytesWritten = RequiredSize;
+fail6:
+fail5:
+    WmiFree(tmppath);
 fail4:
+    FreeUTF8String(&path);
 fail3:
-    XENBUS_STORE(Free, &fdoData->StoreInterface, listresults);
-
 fail2:
-    WmiFree(tmppath);
-
 fail1:
-    FreeUTF8String(&path);
-    *byteswritten = RequiredSize;
     return status;
 }
 
+static NTSTATUS
+SessionExecuteLog(
+    IN  PXENIFACE_FDO   Fdo,
+    IN  PUNICODE_STRING instance,
+    IN  UCHAR*          InBuffer,
+    IN  ULONG           InBufferSize,
+    IN  UCHAR*          OutBuffer,
+    IN  ULONG           OutBufferSize,
+    OUT ULONG_PTR*      BytesWritten
+    )
+{
+    ULONG               RequiredSize;
+    UCHAR*              uloc;
+    NTSTATUS            status;
+    ANSI_STRING         message;
 
-NTSTATUS
-SessionExecuteLog(UCHAR *InBuffer,
-                        ULONG InBufferSize,
-                        UCHAR *OutBuffer,
-                        ULONG OutBufferSize,
-                        XENIFACE_FDO* fdoData,
-                        UNICODE_STRING *instance,
-                        OUT ULONG_PTR *byteswritten) {
-
-    ULONG RequiredSize;
-    UCHAR *uloc;
-    NTSTATUS status;
-    ANSI_STRING message;
+    *BytesWritten = 0;
+    status = STATUS_INVALID_DEVICE_REQUEST;
     if (!AccessWmiBuffer(InBuffer, TRUE, &RequiredSize, InBufferSize,
-                                            WMI_STRING, &uloc,
-                                            WMI_DONE))
-        return STATUS_INVALID_DEVICE_REQUEST;
+                         WMI_STRING, &uloc,
+                         WMI_DONE))
+        goto fail1;
 
     status = GetCountedAnsiString(&message, uloc);
-
     if (!NT_SUCCESS(status))
-        return status;
+        goto fail2;
 
     Info("USER: %s\n", message.Buffer);
 
     RtlFreeAnsiString(&message);
-    *byteswritten = 0;
+
     return STATUS_SUCCESS;
 
+fail2:
+fail1:
+    return status;
 }
 
-NTSTATUS
-SessionExecuteStartTransaction(UCHAR *InBuffer,
-                        ULONG InBufferSize,
-                        UCHAR *OutBuffer,
-                        ULONG OutBufferSize,
-                        XENIFACE_FDO* fdoData,
-                        UNICODE_STRING *instance,
-                        OUT ULONG_PTR *byteswritten) {
-
-    NTSTATUS status = STATUS_SUCCESS;
-    XenStoreSession *session;
-
-    if (!fdoData->InterfacesAcquired) {
-        status= STATUS_INSUFFICIENT_RESOURCES;
-        goto failnotinitialised;
-    }
-    if ((session = FindSessionByInstanceAndLock(fdoData, instance)) ==
-            NULL){
-        status= STATUS_WMI_INSTANCE_NOT_FOUND;
-        goto failsessionnotfound;
-    }
+static NTSTATUS
+SessionExecuteStartTransaction(
+    IN  PXENIFACE_FDO   Fdo,
+    IN  PUNICODE_STRING instance,
+    IN  UCHAR*          InBuffer,
+    IN  ULONG           InBufferSize,
+    IN  UCHAR*          OutBuffer,
+    IN  ULONG           OutBufferSize,
+    OUT ULONG_PTR*      BytesWritten
+    )
+{
+    NTSTATUS            status;
+    PXENSTORE_SESSION   session;
 
-    if (session->transaction!=NULL) {
-        status = STATUS_REQUEST_OUT_OF_SEQUENCE;
-        goto failtransactionactive;
-    }
+    *BytesWritten = 0;
+    status = STATUS_INSUFFICIENT_RESOURCES;
+    if (!Fdo->InterfacesAcquired)
+        goto fail1;
+
+    session = FindSessionByInstanceAndLock(Fdo, instance);
+    status = STATUS_WMI_INSTANCE_NOT_FOUND;
+    if (session == NULL)
+        goto fail2;
 
-    XENBUS_STORE(TransactionStart, &fdoData->StoreInterface, &session->transaction);
+    status = STATUS_REQUEST_OUT_OF_SEQUENCE;
+    if (session->Transaction != NULL)
+        goto fail3;
 
+    XENBUS_STORE(TransactionStart, &Fdo->StoreInterface, &session->Transaction);
 
-failtransactionactive:
-    UnlockSessions(fdoData);
-failsessionnotfound:
-failnotinitialised:
+    ReleaseMutex(&Fdo->SessionLock);
 
-    *byteswritten = 0;
-    return status;
+    return STATUS_SUCCESS;
 
+fail3:
+    ReleaseMutex(&Fdo->SessionLock);
+fail2:
+fail1:
+    return status;
 }
-NTSTATUS
-SessionExecuteCommitTransaction(UCHAR *InBuffer,
-                        ULONG InBufferSize,
-                        UCHAR *OutBuffer,
-                        ULONG OutBufferSize,
-                        XENIFACE_FDO* fdoData,
-                        UNICODE_STRING *instance,
-                        OUT ULONG_PTR *byteswritten) {
-
-    NTSTATUS status = STATUS_SUCCESS;
-    XenStoreSession *session;
-
-    if (!fdoData->InterfacesAcquired) {
-        status= STATUS_INSUFFICIENT_RESOURCES;
-        goto failnotinitialised;
-    }
-    if ((session = FindSessionByInstanceAndLock(fdoData, instance)) ==
-            NULL){
-        status= STATUS_WMI_INSTANCE_NOT_FOUND;
-        goto failsessionnotfound;
-    }
 
-    if (session->transaction==NULL) {
-        status = STATUS_REQUEST_OUT_OF_SEQUENCE;
-        goto failtransactionnotactive;
-    }
+static NTSTATUS
+SessionExecuteCommitTransaction(
+    IN  PXENIFACE_FDO   Fdo,
+    IN  PUNICODE_STRING instance,
+    IN  UCHAR*          InBuffer,
+    IN  ULONG           InBufferSize,
+    IN  UCHAR*          OutBuffer,
+    IN  ULONG           OutBufferSize,
+    OUT ULONG_PTR*      BytesWritten
+    )
+{
+    NTSTATUS            status;
+    PXENSTORE_SESSION   session;
+
+    *BytesWritten = 0;
+    status = STATUS_INSUFFICIENT_RESOURCES;
+    if (!Fdo->InterfacesAcquired)
+        goto fail1;
 
-    status = XENBUS_STORE(TransactionEnd,&fdoData->StoreInterface, session->transaction, TRUE);
+    session = FindSessionByInstanceAndLock(Fdo, instance);
+    status = STATUS_WMI_INSTANCE_NOT_FOUND;
+    if (session == NULL)
+        goto fail2;
+
+    status = STATUS_REQUEST_OUT_OF_SEQUENCE;
+    if (session->Transaction == NULL)
+        goto fail3;
 
-    session->transaction = NULL;
+    status = XENBUS_STORE(TransactionEnd, &Fdo->StoreInterface, session->Transaction, TRUE);
+    session->Transaction = NULL;
 
-failtransactionnotactive:
-    UnlockSessions(fdoData);
-failsessionnotfound:
-failnotinitialised:
+    if (!NT_SUCCESS(status))
+        goto fail4;
 
-    *byteswritten = 0;
-    return status;
+    ReleaseMutex(&Fdo->SessionLock);
 
+    return STATUS_SUCCESS;
+
+fail4:
+fail3:
+    ReleaseMutex(&Fdo->SessionLock);
+fail2:
+fail1:
+    return status;
 }
-NTSTATUS
-SessionExecuteAbortTransaction(UCHAR *InBuffer,
-                        ULONG InBufferSize,
-                        UCHAR *OutBuffer,
-                        ULONG OutBufferSize,
-                        XENIFACE_FDO* fdoData,
-                        UNICODE_STRING *instance,
-                        OUT ULONG_PTR *byteswritten) {
-
-    NTSTATUS status = STATUS_SUCCESS;
-    XenStoreSession *session;
-
-    if (!fdoData->InterfacesAcquired) {
-        status= STATUS_INSUFFICIENT_RESOURCES;
-        goto failnotinitialised;
-    }
-    if ((session = FindSessionByInstanceAndLock(fdoData, instance)) ==
-            NULL){
-        status= STATUS_WMI_INSTANCE_NOT_FOUND;
-        goto failsessionnotfound;
-    }
 
-    if (session->transaction==NULL) {
-        status = STATUS_REQUEST_OUT_OF_SEQUENCE;
-        goto failtransactionnotactive;
-    }
+static NTSTATUS
+SessionExecuteAbortTransaction(
+    IN  PXENIFACE_FDO   Fdo,
+    IN  PUNICODE_STRING instance,
+    IN  UCHAR*          InBuffer,
+    IN  ULONG           InBufferSize,
+    IN  UCHAR*          OutBuffer,
+    IN  ULONG           OutBufferSize,
+    OUT ULONG_PTR*      BytesWritten
+    )
+{
+    NTSTATUS            status;
+    PXENSTORE_SESSION   session;
 
-    status = XENBUS_STORE(TransactionEnd, &fdoData->StoreInterface, session->transaction, FALSE);
+    *BytesWritten = 0;
+    status = STATUS_INSUFFICIENT_RESOURCES;
+    if (!Fdo->InterfacesAcquired)
+        goto fail1;
+
+    session = FindSessionByInstanceAndLock(Fdo, instance);
+    status = STATUS_WMI_INSTANCE_NOT_FOUND;
+    if (session == NULL)
+        goto fail2;
 
-    session->transaction = NULL;
+    status = STATUS_REQUEST_OUT_OF_SEQUENCE;
+    if (session->Transaction == NULL)
+        goto fail3;
 
-failtransactionnotactive:
-    UnlockSessions(fdoData);
-failsessionnotfound:
-failnotinitialised:
+    status = XENBUS_STORE(TransactionEnd, &Fdo->StoreInterface, session->Transaction, FALSE);
+    session->Transaction = NULL;
 
-    *byteswritten = 0;
-    return status;
+    if (!NT_SUCCESS(status))
+        goto fail4;
+
+    ReleaseMutex(&Fdo->SessionLock);
+
+    return STATUS_SUCCESS;
 
+fail4:
+fail3:
+    ReleaseMutex(&Fdo->SessionLock);
+fail2:
+fail1:
+    return status;
 }
 
-NTSTATUS
-SessionExecuteGetValue(UCHAR *InBuffer,
-                        ULONG InBufferSize,
-                        UCHAR *OutBuffer,
-                        ULONG OutBufferSize,
-                        XENIFACE_FDO* fdoData,
-                        UNICODE_STRING *instance,
-                        OUT ULONG_PTR *byteswritten) {
-    NTSTATUS status;
-    OEM_STRING path;
-    UCHAR *uloc;
-    char *value;
-    UCHAR *valuepos;
-    char *tmppath;
-    ULONG RequiredSize;
-    XenStoreSession *session;
-
-    *byteswritten = 0;
+static NTSTATUS
+SessionExecuteGetValue(
+    IN  PXENIFACE_FDO   Fdo,
+    IN  PUNICODE_STRING instance,
+    IN  UCHAR*          InBuffer,
+    IN  ULONG           InBufferSize,
+    IN  UCHAR*          OutBuffer,
+    IN  ULONG           OutBufferSize,
+    OUT ULONG_PTR*      BytesWritten
+    )
+{
+    NTSTATUS            status;
+    OEM_STRING          path;
+    UCHAR*              uloc;
+    char*               value;
+    UCHAR*              valuepos;
+    char*               tmppath;
+    ULONG               RequiredSize;
+    PXENSTORE_SESSION   session;
+
+    *BytesWritten = 0;
+    status = STATUS_INVALID_DEVICE_REQUEST;
     if (!AccessWmiBuffer(InBuffer, TRUE, &RequiredSize, InBufferSize,
-                                            WMI_STRING, &uloc,
-                                            WMI_DONE))
-        return STATUS_INVALID_DEVICE_REQUEST;
-    if (!fdoData->InterfacesAcquired) {
-        return STATUS_INSUFFICIENT_RESOURCES;
-    }
+                         WMI_STRING, &uloc,
+                         WMI_DONE))
+        goto fail1;
 
-    status = GetCountedUTF8String(&path, uloc);
+    status = STATUS_INSUFFICIENT_RESOURCES;
+    if (!Fdo->InterfacesAcquired)
+        goto fail2;
 
+    status = GetCountedUTF8String(&path, uloc);
     if (!NT_SUCCESS(status))
-        return status;;
+        goto fail3;
 
     status = STATUS_INSUFFICIENT_RESOURCES;
     tmppath = WmiAllocate(path.Length + 1);
-    if (!tmppath) {
-        goto fail1;
-    }
+    if (tmppath == NULL)
+        goto fail4;
 
     RtlCopyBytes(tmppath, path.Buffer, path.Length);
 
     status = STATUS_WMI_INSTANCE_NOT_FOUND;
-    if ((session = FindSessionByInstanceAndLock(fdoData, instance)) ==
-            NULL){
-        goto fail2;
-    }
-    status = XENBUS_STORE(Read, &fdoData->StoreInterface, session->transaction, NULL, tmppath, &value);
-    UnlockSessions(fdoData);
+    session = FindSessionByInstanceAndLock(Fdo, instance);
+    if (session == NULL)
+        goto fail5;
+
+    status = XENBUS_STORE(Read, &Fdo->StoreInterface, session->Transaction, NULL, tmppath, &value);
+    ReleaseMutex(&Fdo->SessionLock);
 
     if (!NT_SUCCESS(status))
-        goto fail2;
+        goto fail6;
 
     status = STATUS_BUFFER_TOO_SMALL;
     if (!AccessWmiBuffer(OutBuffer, FALSE, &RequiredSize, OutBufferSize,
-                            WMI_STRING, GetCountedUtf8Size(value), &valuepos,
-                            WMI_DONE)) {
-        goto fail3;
-    }
-    status = STATUS_SUCCESS;
-    WriteCountedUTF8String(value, valuepos);
+                         WMI_STRING, GetCountedUtf8Size(value), &valuepos,
+                         WMI_DONE))
+        goto fail7;
 
-fail3:
-    XENBUS_STORE(Free, &fdoData->StoreInterface, value);
-    *byteswritten = RequiredSize;
+    WriteCountedUTF8String(value, valuepos);
 
-fail2:
+    XENBUS_STORE(Free, &Fdo->StoreInterface, value);
     WmiFree(tmppath);
+    FreeUTF8String(&path);
 
-fail1:
+    *BytesWritten = RequiredSize;
+    return STATUS_SUCCESS;
+
+fail7:
+    XENBUS_STORE(Free, &Fdo->StoreInterface, value);
+    *BytesWritten = RequiredSize;
+fail6:
+fail5:
+    WmiFree(tmppath);
+fail4:
     FreeUTF8String(&path);
+fail3:
+fail2:
+fail1:
     return status;
 }
-NTSTATUS
-BaseExecuteAddSession(UCHAR *InBuffer,
-                        ULONG InBufferSize,
-                        UCHAR *OutBuffer,
-                        ULONG OutBufferSize,
-                        XENIFACE_FDO* fdoData,
-                        OUT ULONG_PTR *byteswritten) {
-    ULONG RequiredSize;
-    UNICODE_STRING ustring;
-    ULONG *id;
-    UCHAR* stringid;
-    NTSTATUS status;
-    *byteswritten = 0;
+
+static NTSTATUS
+BaseExecuteAddSession(
+    IN  PXENIFACE_FDO   Fdo,
+    IN  UCHAR*          InBuffer,
+    IN  ULONG           InBufferSize,
+    IN  UCHAR*          OutBuffer,
+    IN  ULONG           OutBufferSize,
+    OUT ULONG_PTR*      BytesWritten
+    )
+{
+    ULONG               RequiredSize;
+    UNICODE_STRING      ustring;
+    ULONG*              id;
+    UCHAR*              StringId;
+    NTSTATUS            status;
+
+    *BytesWritten = 0;
+    status = STATUS_INVALID_DEVICE_REQUEST;
     if (!AccessWmiBuffer(InBuffer, TRUE, &RequiredSize, InBufferSize,
-                            WMI_STRING, &stringid,
-                            WMI_DONE)){
-        return STATUS_INVALID_DEVICE_REQUEST;
-    }
+                         WMI_STRING, &StringId,
+                         WMI_DONE))
+        goto fail1;
+
+    status = STATUS_BUFFER_TOO_SMALL;
     if (!AccessWmiBuffer(OutBuffer, FALSE, &RequiredSize, OutBufferSize,
-                            WMI_UINT32, &id,
-                            WMI_DONE)) {
-        *byteswritten = RequiredSize;
-        return STATUS_BUFFER_TOO_SMALL;
-    }
+                         WMI_UINT32, &id,
+                         WMI_DONE))
+        goto fail2;
 
-    AllocUnicodeStringBuffer(&ustring, *(USHORT*)(stringid));
+    AllocUnicodeStringBuffer(&ustring, *(USHORT*)(StringId));
+    status = STATUS_INSUFFICIENT_RESOURCES;
     if (ustring.Buffer == NULL)
-        return STATUS_INSUFFICIENT_RESOURCES;
+        goto fail3;
+
     status = RtlUnicodeStringCbCopyStringN(&ustring,
-                                            (LPCWSTR)(stringid+sizeof(USHORT)),
-                                            *(USHORT*)(stringid));
-    if (!NT_SUCCESS(status)) {
-        FreeUnicodeStringBuffer(&ustring);
-        return status;
-    }
-    status = CreateNewSession(fdoData, &ustring, id);
-    if (!NT_SUCCESS(status)) {
-        FreeUnicodeStringBuffer(&ustring);
-        return status;
-    }
+                                           (LPCWSTR)(StringId + sizeof(USHORT)),
+                                           *(USHORT*)(StringId));
+    if (!NT_SUCCESS(status))
+        goto fail4;
+
+    status = SessionCreate(Fdo, &ustring, id);
+    if (!NT_SUCCESS(status))
+        goto fail5;
 
-    *byteswritten = RequiredSize;
+    *BytesWritten = RequiredSize;
     return STATUS_SUCCESS;
 
+fail5:
+fail4:
+    FreeUnicodeStringBuffer(&ustring);
+fail3:
+fail2:
+    *BytesWritten = RequiredSize;
+fail1:
+    return status;
 }
 
+static NTSTATUS
+SessionExecuteMethod(
+    IN  PXENIFACE_FDO   Fdo,
+    IN  UCHAR*          Buffer,
+    IN  ULONG           BufferSize,
+    OUT ULONG_PTR*      BytesWritten
+    )
+{
+    ULONG               RequiredSize;
+    WNODE_METHOD_ITEM*  Method;
+    UCHAR*              InBuffer;
+    NTSTATUS            status;
+    UNICODE_STRING      instance;
+    UCHAR*              InstStr;
 
-NTSTATUS
-SessionExecuteMethod(UCHAR *Buffer,
-                    ULONG BufferSize,
-                    XENIFACE_FDO* fdoData,
-                    OUT ULONG_PTR *byteswritten) {
-    ULONG RequiredSize;
-    WNODE_METHOD_ITEM *Method;
-    UCHAR *InBuffer;
-    NTSTATUS status;
-    UNICODE_STRING instance;
-    UCHAR *InstStr;
-    Trace("%s\n",__FUNCTION__);
     if (!AccessWmiBuffer(Buffer, TRUE, &RequiredSize, BufferSize,
-                            WMI_BUFFER, sizeof(WNODE_METHOD_ITEM),
-                                &Method,
-                            WMI_DONE))
-    {
+                         WMI_BUFFER, sizeof(WNODE_METHOD_ITEM), &Method,
+                         WMI_DONE))
         return STATUS_INVALID_DEVICE_REQUEST;
-    }
+
     if (!AccessWmiBuffer(Buffer, TRUE, &RequiredSize, BufferSize,
-                            WMI_BUFFER, sizeof(WNODE_METHOD_ITEM),
-                                &Method,
-                            WMI_STRINGOFFSET, Method->OffsetInstanceName,
-                                &InstStr,
-                            WMI_DONE))
-    {
+                         WMI_BUFFER, sizeof(WNODE_METHOD_ITEM), &Method,
+                         WMI_STRINGOFFSET, Method->OffsetInstanceName, &InstStr,
+                         WMI_DONE))
         return STATUS_INVALID_DEVICE_REQUEST;
-    }
 
     InBuffer = Buffer + Method->DataBlockOffset;
 
     GetCountedUnicodeString(&instance, InstStr);
 
-
-    Trace("Method Id %d\n", Method->MethodId);
     switch (Method->MethodId) {
-        case GetValue:
-            status = SessionExecuteGetValue(InBuffer, Method->SizeDataBlock,
-                                              Buffer+Method->DataBlockOffset,
-                                              BufferSize-Method->DataBlockOffset,
-                                              fdoData,
-                                              &instance,
-                                              byteswritten);
-            break;
-        case SetValue:
-            status = SessionExecuteSetValue(InBuffer, Method->SizeDataBlock,
-                                              Buffer+Method->DataBlockOffset,
-                                              BufferSize-Method->DataBlockOffset,
-                                              fdoData,
-                                              &instance,
-                                              byteswritten);
-            break;
-        case GetChildren:
-            status = SessionExecuteGetChildren(InBuffer, Method->SizeDataBlock,
-                                              Buffer+Method->DataBlockOffset,
-                                              BufferSize-Method->DataBlockOffset,
-                                              fdoData,
-                                              &instance,
-                                              byteswritten);
-            break;
-        case SetWatch:
-            status = SessionExecuteSetWatch(InBuffer, Method->SizeDataBlock,
-                                              Buffer+Method->DataBlockOffset,
-                                              BufferSize-Method->DataBlockOffset,
-                                              fdoData,
-                                              &instance,
-                                              byteswritten);
-            break;
-        case EndSession:
-            status = SessionExecuteEndSession(InBuffer, Method->SizeDataBlock,
-                                              Buffer+Method->DataBlockOffset,
-                                              BufferSize-Method->DataBlockOffset,
-                                              fdoData,
-                                              &instance,
-                                              byteswritten);
-            break;
-        case RemoveWatch:
-            status = SessionExecuteRemoveWatch(InBuffer, Method->SizeDataBlock,
-                                              Buffer+Method->DataBlockOffset,
-                                              BufferSize-Method->DataBlockOffset,
-                                              fdoData,
+    case GetValue:
+        status = SessionExecuteGetValue(Fdo,
+                                        &instance,
+                                        InBuffer,
+                                        Method->SizeDataBlock,
+                                        Buffer + Method->DataBlockOffset,
+                                        BufferSize - Method->DataBlockOffset,
+                                        BytesWritten);
+        break;
+    case SetValue:
+        status = SessionExecuteSetValue(Fdo,
+                                        &instance,
+                                        InBuffer,
+                                        Method->SizeDataBlock,
+                                        Buffer + Method->DataBlockOffset,
+                                        BufferSize - Method->DataBlockOffset,
+                                        BytesWritten);
+        break;
+    case GetChildren:
+        status = SessionExecuteGetChildren(Fdo,
+                                           &instance,
+                                           InBuffer,
+                                           Method->SizeDataBlock,
+                                           Buffer + Method->DataBlockOffset,
+                                           BufferSize - Method->DataBlockOffset,
+                                           BytesWritten);
+        break;
+    case SetWatch:
+        status = SessionExecuteSetWatch(Fdo,
+                                        &instance,
+                                        InBuffer, Method->SizeDataBlock,
+                                        Buffer + Method->DataBlockOffset,
+                                        BufferSize - Method->DataBlockOffset,
+                                        BytesWritten);
+        break;
+    case EndSession:
+        status = SessionExecuteEndSession(Fdo,
+                                          &instance,
+                                          InBuffer,
+                                          Method->SizeDataBlock,
+                                          Buffer + Method->DataBlockOffset,
+                                          BufferSize - Method->DataBlockOffset,
+                                          BytesWritten);
+        break;
+    case RemoveWatch:
+        status = SessionExecuteRemoveWatch(Fdo,
+                                           &instance,
+                                           InBuffer,
+                                           Method->SizeDataBlock,
+                                           Buffer + Method->DataBlockOffset,
+                                           BufferSize - Method->DataBlockOffset,
+                                           BytesWritten);
+        break;
+    case RemoveValue:
+        status = SessionExecuteRemoveValue(Fdo,
+                                           &instance,
+                                           InBuffer,
+                                           Method->SizeDataBlock,
+                                           Buffer + Method->DataBlockOffset,
+                                           BufferSize - Method->DataBlockOffset,
+                                           BytesWritten);
+        break;
+    case Log:
+        status = SessionExecuteLog(Fdo,
+                                   &instance,
+                                   InBuffer,
+                                   Method->SizeDataBlock,
+                                   Buffer + Method->DataBlockOffset,
+                                   BufferSize - Method->DataBlockOffset,
+                                   BytesWritten);
+        break;
+    case StartTransaction:
+        status = SessionExecuteStartTransaction(Fdo,
+                                                &instance,
+                                                InBuffer,
+                                                Method->SizeDataBlock,
+                                                Buffer + Method->DataBlockOffset,
+                                                BufferSize - Method->DataBlockOffset,
+                                                BytesWritten);
+        break;
+    case CommitTransaction:
+        status = SessionExecuteCommitTransaction(Fdo,
+                                                 &instance,
+                                                 InBuffer,
+                                                 Method->SizeDataBlock,
+                                                 Buffer + Method->DataBlockOffset,
+                                                 BufferSize - Method->DataBlockOffset,
+                                                 BytesWritten);
+        break;
+    case AbortTransaction:
+        status = SessionExecuteAbortTransaction(Fdo,
+                                                &instance,
+                                                InBuffer,
+                                                Method->SizeDataBlock,
+                                                Buffer + Method->DataBlockOffset,
+                                                BufferSize - Method->DataBlockOffset,
+                                                BytesWritten);
+        break;
+    case GetFirstChild:
+        status = SessionExecuteGetFirstChild(Fdo,
+                                             &instance,
+                                             InBuffer,
+                                             Method->SizeDataBlock,
+                                             Buffer + Method->DataBlockOffset,
+                                             BufferSize - Method->DataBlockOffset,
+                                             BytesWritten);
+        break;
+    case GetNextSibling:
+        status = SessionExecuteGetNextSibling(Fdo,
                                               &instance,
-                                              byteswritten);
-            break;
-        case RemoveValue:
-            status = SessionExecuteRemoveValue(InBuffer, Method->SizeDataBlock,
-                                              Buffer+Method->DataBlockOffset,
-                                              BufferSize-Method->DataBlockOffset,
-                                              fdoData,
-                                              &instance,
-                                              byteswritten);
-            break;
-        case Log:
-            status = SessionExecuteLog(InBuffer,  Method->SizeDataBlock,
-                                              Buffer+Method->DataBlockOffset,
-                                              BufferSize-Method->DataBlockOffset,
-                                              fdoData,
-                                              &instance,
-                                              byteswritten);
-            break;
-        case StartTransaction:
-            status = SessionExecuteStartTransaction(InBuffer,  Method->SizeDataBlock,
-                                              Buffer+Method->DataBlockOffset,
-                                              BufferSize-Method->DataBlockOffset,
-                                              fdoData,
-                                              &instance,
-                                              byteswritten);
-            break;
-        case CommitTransaction:
-            status = SessionExecuteCommitTransaction(InBuffer,  Method->SizeDataBlock,
-                                              Buffer+Method->DataBlockOffset,
-                                              BufferSize-Method->DataBlockOffset,
-                                              fdoData,
-                                              &instance,
-                                              byteswritten);
-            break;
-        case AbortTransaction:
-            status = SessionExecuteAbortTransaction(InBuffer,  Method->SizeDataBlock,
-                                              Buffer+Method->DataBlockOffset,
-                                              BufferSize-Method->DataBlockOffset,
-                                              fdoData,
-                                              &instance,
-                                              byteswritten);
-            break;
-        case GetFirstChild:
-            status = SessionExecuteGetFirstChild(InBuffer,  Method->SizeDataBlock,
-                                              Buffer+Method->DataBlockOffset,
-                                              BufferSize-Method->DataBlockOffset,
-                                              fdoData,
-                                              &instance,
-                                              byteswritten);
-            break;
-        case GetNextSibling:
-            status = SessionExecuteGetNextSibling(InBuffer,  Method->SizeDataBlock,
-                                              Buffer+Method->DataBlockOffset,
-                                              BufferSize-Method->DataBlockOffset,
-                                              fdoData,
-                                              &instance,
-                                              byteswritten);
-            break;
-
-
-        default:
-            Info("DRV: Unknown WMI method %d\n", Method->MethodId);
-            return STATUS_WMI_ITEMID_NOT_FOUND;
-    }
-    Method->SizeDataBlock = (ULONG)*byteswritten;
-    *byteswritten+=Method->DataBlockOffset;
-    if (status == STATUS_BUFFER_TOO_SMALL) {
-        return NodeTooSmall(Buffer, BufferSize, (ULONG)*byteswritten, byteswritten);
+                                              InBuffer,
+                                              Method->SizeDataBlock,
+                                              Buffer + Method->DataBlockOffset,
+                                              BufferSize - Method->DataBlockOffset,
+                                              BytesWritten);
+        break;
+    default:
+        Info("DRV: Unknown WMI method %d\n", Method->MethodId);
+        return STATUS_WMI_ITEMID_NOT_FOUND;
     }
 
-    Method->WnodeHeader.BufferSize = (ULONG)*byteswritten;
-     return status;
+    Method->SizeDataBlock = (ULONG)*BytesWritten;
+    *BytesWritten += Method->DataBlockOffset;
+    if (status == STATUS_BUFFER_TOO_SMALL)
+        return NodeTooSmall(Buffer, BufferSize, (ULONG)*BytesWritten, BytesWritten);
+
+    Method->WnodeHeader.BufferSize = (ULONG)*BytesWritten;
+    return status;
 }
-NTSTATUS
-BaseExecuteMethod(UCHAR *Buffer,
-                    ULONG BufferSize,
-                    XENIFACE_FDO* fdoData,
-                    OUT ULONG_PTR *byteswritten) {
-    ULONG RequiredSize;
-    WNODE_METHOD_ITEM *Method;
-    UCHAR *InBuffer;
-    NTSTATUS status;
+
+static NTSTATUS
+BaseExecuteMethod(
+    IN  PXENIFACE_FDO       Fdo,
+    IN  UCHAR*              Buffer,
+    IN  ULONG               BufferSize,
+    OUT ULONG_PTR*          BytesWritten
+    )
+{
+    ULONG                   RequiredSize;
+    WNODE_METHOD_ITEM*      Method;
+    UCHAR*                  InBuffer;
+    NTSTATUS                status;
+
     if (!AccessWmiBuffer(Buffer, TRUE, &RequiredSize, BufferSize,
-                            WMI_BUFFER, sizeof(WNODE_METHOD_ITEM),
-                                &Method,
-                            WMI_DONE))
-    {
+                         WMI_BUFFER, sizeof(WNODE_METHOD_ITEM), &Method,
+                         WMI_DONE))
         return STATUS_INVALID_DEVICE_REQUEST;
-    }
 
     InBuffer = Buffer + Method->DataBlockOffset;
 
     switch (Method->MethodId) {
-        case AddSession:
-            status = BaseExecuteAddSession(InBuffer, Method->SizeDataBlock,
-                                             Buffer+Method->DataBlockOffset,
-                                             BufferSize-Method->DataBlockOffset,
-                                             fdoData,
-                                             byteswritten);
-            Method->SizeDataBlock = (ULONG)*byteswritten;
-            *byteswritten+=Method->DataBlockOffset;
-            Method->WnodeHeader.BufferSize = (ULONG)*byteswritten;
-            return status;
+    case AddSession:
+        status = BaseExecuteAddSession(Fdo,
+                                       InBuffer,
+                                       Method->SizeDataBlock,
+                                       Buffer + Method->DataBlockOffset,
+                                       BufferSize - Method->DataBlockOffset,
+                                       BytesWritten);
+        break;
 
-        default:
-            return STATUS_WMI_ITEMID_NOT_FOUND;
+    default:
+        return STATUS_WMI_ITEMID_NOT_FOUND;
     }
+
+    Method->SizeDataBlock = (ULONG)*BytesWritten;
+    *BytesWritten += Method->DataBlockOffset;
+    Method->WnodeHeader.BufferSize = (ULONG)*BytesWritten;
+
+    return status;
 }
 
-NTSTATUS
+static NTSTATUS
 WmiExecuteMethod(
-    IN PXENIFACE_FDO fdoData,
-    IN PIO_STACK_LOCATION stack,
-    OUT ULONG_PTR *byteswritten
-   )
+    IN  PXENIFACE_FDO       Fdo,
+    IN  PIO_STACK_LOCATION  Stack,
+    OUT ULONG_PTR*          BytesWritten
+    )
 {
-    if (IsEqualGUID(stack->Parameters.WMI.DataPath,
-                    &OBJECT_GUID(XenStoreBase))) {
-        return BaseExecuteMethod(stack->Parameters.WMI.Buffer,
-                                    stack->Parameters.WMI.BufferSize,
-                                    fdoData,  byteswritten);
-    }
-    else if (IsEqualGUID(stack->Parameters.WMI.DataPath,
-                         &OBJECT_GUID(XenStoreSession))) {
-        return SessionExecuteMethod(stack->Parameters.WMI.Buffer,
-                                    stack->Parameters.WMI.BufferSize,
-                                    fdoData,  byteswritten);
-    }
+    if (IsEqualGUID(Stack->Parameters.WMI.DataPath,
+                    &OBJECT_GUID(XenStoreBase)))
+        return BaseExecuteMethod(Fdo,
+                                 Stack->Parameters.WMI.Buffer,
+                                 Stack->Parameters.WMI.BufferSize,
+                                 BytesWritten);
+
+    if (IsEqualGUID(Stack->Parameters.WMI.DataPath,
+                    &OBJECT_GUID(XenStoreSession)))
+        return SessionExecuteMethod(Fdo,
+                                    Stack->Parameters.WMI.Buffer,
+                                    Stack->Parameters.WMI.BufferSize,
+                                    BytesWritten);
 
-    else
-        return STATUS_NOT_SUPPORTED;
+    return STATUS_NOT_SUPPORTED;
 }
 
-NTSTATUS
-GenerateSessionBlock(UCHAR *Buffer,
-                        ULONG BufferSize,
-                        PXENIFACE_FDO fdoData,
-                        ULONG_PTR *byteswritten) {
-    WNODE_ALL_DATA *node;
-    ULONG RequiredSize;
-    size_t nodesizerequired;
-    size_t namesizerequired;
-    int entries;
-    XenStoreSession *session;
+static NTSTATUS
+GenerateSessionBlock(
+    IN  PXENIFACE_FDO       Fdo,
+    IN  UCHAR*              Buffer,
+    IN  ULONG               BufferSize,
+    OUT ULONG_PTR*          BytesWritten
+    )
+{
+    PLIST_ENTRY             ListEntry;
+    PXENSTORE_SESSION       Session;
+    WNODE_ALL_DATA*         node;
+    ULONG                   RequiredSize;
+    size_t                  nodesizerequired;
+    size_t                  namesizerequired;
+    int                     entries;
     OFFSETINSTANCEDATAANDLENGTH* dataoffsets;
-    ULONG* nameoffsets;
-    UCHAR *data;
-    UCHAR *names;
+    ULONG*                  nameoffsets;
+    UCHAR*                  data;
+    UCHAR*                  names;
+    int                     entrynum = 0;
+    UCHAR*                  datapos;
+    UCHAR*                  namepos;
 
-
-    LockSessions(fdoData);
+    AcquireMutex(&Fdo->SessionLock);
 
     //work out how much space we need for each session structure
     nodesizerequired = 0;
     namesizerequired = 0;
     entries = 0;
-    session = (XenStoreSession *)fdoData->SessionHead.Flink;
-    //work out names for each session entry
-    while (session !=  (XenStoreSession *)&fdoData->SessionHead) {
+
+    for (ListEntry = Fdo->SessionHead.Flink;
+         ListEntry != &Fdo->SessionHead;
+         ListEntry = ListEntry->Flink) {
         ULONG *id;
         UCHAR *sesbuf;
         UCHAR *inamebuf;
 
+        Session = CONTAINING_RECORD(ListEntry, XENSTORE_SESSION, ListEntry);
+
         (VOID) AccessWmiBuffer((PUCHAR)nodesizerequired, FALSE, &RequiredSize, 0,
                         WMI_UINT32, &id,
-                        WMI_STRING,
-                            GetCountedUnicodeStringSize(&session->stringid),
-                            &sesbuf,
+                        WMI_STRING, GetCountedUnicodeStringSize(&Session->StringId), &sesbuf,
                         WMI_DONE);
         nodesizerequired += RequiredSize;
 
         (VOID) AccessWmiBuffer((PUCHAR)namesizerequired, FALSE, &RequiredSize, 0,
-                        WMI_STRING,
-                            GetCountedUnicodeStringSize(&session->instancename),
-                            &inamebuf,
+                        WMI_STRING, GetCountedUnicodeStringSize(&Session->InstanceName), &inamebuf,
                         WMI_DONE);
         namesizerequired += RequiredSize;
         entries++;
-        session = (XenStoreSession *)session->listentry.Flink;
     }
 
-    //perform the access check
+    // perform the access check
     if (!AccessWmiBuffer(Buffer, FALSE, &RequiredSize, BufferSize,
-                            WMI_BUFFER, sizeof(WNODE_ALL_DATA), &node,
-                            WMI_BUFFER, sizeof(OFFSETINSTANCEDATAANDLENGTH)*
-                                            entries, &dataoffsets,
-                            WMI_BUFFER, sizeof(ULONG)*entries, &nameoffsets,
-                            WMI_BUFFER, nodesizerequired, &data,
-                            WMI_BUFFER, namesizerequired, &names,
-                            WMI_DONE)) {
-        UnlockSessions(fdoData);
-        return NodeTooSmall(Buffer, BufferSize, RequiredSize, byteswritten);
+                         WMI_BUFFER, sizeof(WNODE_ALL_DATA), &node,
+                         WMI_BUFFER, sizeof(OFFSETINSTANCEDATAANDLENGTH) * entries, &dataoffsets,
+                         WMI_BUFFER, sizeof(ULONG)*entries, &nameoffsets,
+                         WMI_BUFFER, nodesizerequired, &data,
+                         WMI_BUFFER, namesizerequired, &names,
+                         WMI_DONE)) {
+        ReleaseMutex(&Fdo->SessionLock);
+        return NodeTooSmall(Buffer, BufferSize, RequiredSize, BytesWritten);
     }
 
     node->DataBlockOffset = (ULONG)(data - Buffer);
@@ -2708,303 +2680,300 @@ GenerateSessionBlock(UCHAR *Buffer,
     KeQuerySystemTime(&node->WnodeHeader.TimeStamp);
     node->WnodeHeader.Flags = WNODE_FLAG_ALL_DATA;
     node->InstanceCount = entries;
-    *byteswritten = RequiredSize;
-
-    session = (XenStoreSession *)fdoData->SessionHead.Flink;
-    {
-        int entrynum = 0;
-        UCHAR *datapos = data;
-        UCHAR *namepos = names;
-        //work out names for each session entry
-        while (session !=  (XenStoreSession *)&fdoData->SessionHead){
-            ULONG *id;
-            UCHAR *sesbuf;
-            UCHAR *inamebuf;
-
-            (VOID) AccessWmiBuffer(datapos, FALSE, &RequiredSize, BufferSize+Buffer-datapos,
-                            WMI_UINT32, &id,
-                            WMI_STRING,
-                                GetCountedUnicodeStringSize(&session->stringid),
-                                &sesbuf,
-                            WMI_DONE);
-
-            node->OffsetInstanceDataAndLength[entrynum].OffsetInstanceData =
-                (ULONG)((UCHAR *)id - Buffer);
-            node->OffsetInstanceDataAndLength[entrynum].LengthInstanceData =
-                RequiredSize;
-            *id = session->id;
-            WriteCountedUnicodeString(&session->stringid, sesbuf);
-            datapos+=RequiredSize;
-
-            (VOID) AccessWmiBuffer(namepos, FALSE, &RequiredSize, BufferSize+Buffer-namepos,
-                            WMI_STRING,
-                                GetCountedUnicodeStringSize(&session->instancename),
-                                &inamebuf,
-                            WMI_DONE);
-
-            nameoffsets[entrynum] = (ULONG)(namepos-Buffer);
-            WriteCountedUnicodeString(&session->instancename, inamebuf);
-            namepos+=RequiredSize;
-
-            namesizerequired += RequiredSize;
-            entrynum++;
-            session = (XenStoreSession *)session->listentry.Flink;
-        }
 
+    *BytesWritten = RequiredSize;
+
+    datapos = data;
+    namepos = names;
+
+    //work out names for each session entry
+    for (ListEntry = Fdo->SessionHead.Flink;
+         ListEntry != &Fdo->SessionHead;
+         ListEntry = ListEntry->Flink) {
+        ULONG *id;
+        UCHAR *sesbuf;
+        UCHAR *inamebuf;
+
+        Session = CONTAINING_RECORD(ListEntry, XENSTORE_SESSION, ListEntry);
+
+        (VOID) AccessWmiBuffer(datapos, FALSE, &RequiredSize, BufferSize+Buffer-datapos,
+                        WMI_UINT32, &id,
+                        WMI_STRING, GetCountedUnicodeStringSize(&Session->StringId), &sesbuf,
+                        WMI_DONE);
+
+        node->OffsetInstanceDataAndLength[entrynum].OffsetInstanceData =
+            (ULONG)((UCHAR *)id - Buffer);
+        node->OffsetInstanceDataAndLength[entrynum].LengthInstanceData =
+            RequiredSize;
+        *id = Session->SessionId;
+        WriteCountedUnicodeString(&Session->StringId, sesbuf);
+        datapos += RequiredSize;
+
+        (VOID) AccessWmiBuffer(namepos, FALSE, &RequiredSize, BufferSize+Buffer-namepos,
+                        WMI_STRING, GetCountedUnicodeStringSize(&Session->InstanceName), &inamebuf,
+                        WMI_DONE);
+
+        nameoffsets[entrynum] = (ULONG)(namepos-Buffer);
+        WriteCountedUnicodeString(&Session->InstanceName, inamebuf);
+        namepos += RequiredSize;
+
+        namesizerequired += RequiredSize;
+        entrynum++;
     }
 
-    UnlockSessions(fdoData);
+    ReleaseMutex(&Fdo->SessionLock);
 
     return STATUS_SUCCESS;
-
 }
 
-NTSTATUS
-GenerateBaseBlock(  XENIFACE_FDO *fdoData,
-                    UCHAR *Buffer,
-                    ULONG BufferSize,
-                    ULONG_PTR *byteswritten) {
-    WNODE_ALL_DATA *node;
-    ULONG RequiredSize;
-    ULONGLONG *time;
+static NTSTATUS
+GenerateBaseBlock(
+    IN  PXENIFACE_FDO       Fdo,
+    IN  UCHAR*              Buffer,
+    IN  ULONG               BufferSize,
+    OUT ULONG_PTR*          BytesWritten
+    )
+{
+    WNODE_ALL_DATA*         node;
+    ULONG                   RequiredSize;
+    ULONGLONG*              time;
+
     if (!AccessWmiBuffer(Buffer, FALSE, &RequiredSize, BufferSize,
-                            WMI_BUFFER, sizeof(WNODE_ALL_DATA), &node,
-                            WMI_UINT64, &time,
-                            WMI_DONE))
-    {
-        return NodeTooSmall(Buffer, BufferSize, RequiredSize, byteswritten);
-    }
+                         WMI_BUFFER, sizeof(WNODE_ALL_DATA), &node,
+                         WMI_UINT64, &time,
+                         WMI_DONE))
+        return NodeTooSmall(Buffer, BufferSize, RequiredSize, BytesWritten);
 
-    node->DataBlockOffset =(ULONG)( ((UCHAR *)time)-Buffer);
+    node->DataBlockOffset = (ULONG)(((UCHAR *)time) - Buffer);
     node->WnodeHeader.BufferSize = RequiredSize;
     KeQuerySystemTime(&node->WnodeHeader.TimeStamp);
     node->WnodeHeader.Flags = WNODE_FLAG_ALL_DATA |
-                                WNODE_FLAG_FIXED_INSTANCE_SIZE |
-                                WNODE_FLAG_PDO_INSTANCE_NAMES;
-    if (fdoData->InterfacesAcquired) {
+                              WNODE_FLAG_FIXED_INSTANCE_SIZE |
+                              WNODE_FLAG_PDO_INSTANCE_NAMES;
+    if (Fdo->InterfacesAcquired) {
         LARGE_INTEGER info;
 
-        XENBUS_SHARED_INFO(GetTime, &fdoData->SharedInfoInterface, &info, NULL);
+        XENBUS_SHARED_INFO(GetTime, &Fdo->SharedInfoInterface, &info, NULL);
         *time = info.QuadPart;
-    }
-    else {
+    } else {
         *time = 0;
     }
+
     node->InstanceCount = 1;
     node->FixedInstanceSize = sizeof(ULONGLONG);
-    *byteswritten = RequiredSize;
+
+    *BytesWritten = RequiredSize;
     return STATUS_SUCCESS;
 }
-NTSTATUS
+
+static NTSTATUS
 GenerateBaseInstance(
-                    XENIFACE_FDO *fdoData,
-                    UCHAR *Buffer,
-                    ULONG BufferSize,
-                    ULONG_PTR *byteswritten) {
-    WNODE_SINGLE_INSTANCE *node;
-    ULONG RequiredSize;
-    ULONGLONG *time;
-    UCHAR * dbo;
-    if (!AccessWmiBuffer(Buffer, FALSE, &RequiredSize, BufferSize,
-                            WMI_BUFFER, sizeof(WNODE_SINGLE_INSTANCE), &node,
-                            WMI_DONE))
-    {
-        return NodeTooSmall(Buffer, BufferSize, RequiredSize, byteswritten);
-    }
+    IN  PXENIFACE_FDO       Fdo,
+    IN  UCHAR*              Buffer,
+    IN  ULONG               BufferSize,
+    OUT ULONG_PTR*          BytesWritten
+    )
+{
+    WNODE_SINGLE_INSTANCE*  node;
+    ULONG                   RequiredSize;
+    ULONGLONG*              time;
+    UCHAR*                  dbo;
+
     if (!AccessWmiBuffer(Buffer, FALSE, &RequiredSize, BufferSize,
-                            WMI_BUFFER, sizeof(WNODE_SINGLE_INSTANCE), &node,
-                            WMI_OFFSET, node->DataBlockOffset, 0 ,&dbo,
-                            WMI_DONE))
-    {
-        return NodeTooSmall(Buffer, BufferSize, RequiredSize, byteswritten);
-    }
-    if (!AccessWmiBuffer(dbo, FALSE, &RequiredSize, BufferSize-node->DataBlockOffset,
-                            WMI_UINT64, &time,
-                            WMI_DONE)){
-        return NodeTooSmall(Buffer, BufferSize, RequiredSize+node->DataBlockOffset,
-                            byteswritten);
-    }
+                         WMI_BUFFER, sizeof(WNODE_SINGLE_INSTANCE), &node,
+                         WMI_DONE))
+        return NodeTooSmall(Buffer, BufferSize, RequiredSize, BytesWritten);
 
-    if (node->InstanceIndex != 0) {
+    if (!AccessWmiBuffer(Buffer, FALSE, &RequiredSize, BufferSize,
+                         WMI_BUFFER, sizeof(WNODE_SINGLE_INSTANCE), &node,
+                         WMI_OFFSET, node->DataBlockOffset, 0 ,&dbo,
+                         WMI_DONE))
+        return NodeTooSmall(Buffer, BufferSize, RequiredSize, BytesWritten);
+
+    if (!AccessWmiBuffer(dbo, FALSE, &RequiredSize, BufferSize - node->DataBlockOffset,
+                         WMI_UINT64, &time,
+                         WMI_DONE))
+        return NodeTooSmall(Buffer,
+                            BufferSize,
+                            RequiredSize + node->DataBlockOffset,
+                            BytesWritten);
+
+    if (node->InstanceIndex != 0)
         return STATUS_WMI_ITEMID_NOT_FOUND;
-    }
-    if (fdoData->InterfacesAcquired) {
+
+    if (Fdo->InterfacesAcquired) {
         LARGE_INTEGER info;
 
-        XENBUS_SHARED_INFO(GetTime, &fdoData->SharedInfoInterface, &info, NULL);
+        XENBUS_SHARED_INFO(GetTime, &Fdo->SharedInfoInterface, &info, NULL);
         *time = info.QuadPart;
-    }
-    else {
+    } else {
         *time = 0;
     }
 
-
-    node->WnodeHeader.BufferSize = node->DataBlockOffset+RequiredSize;
+    node->WnodeHeader.BufferSize = node->DataBlockOffset + RequiredSize;
     node->SizeDataBlock = RequiredSize;
 
-    *byteswritten = node->DataBlockOffset+RequiredSize;
+    *BytesWritten = node->DataBlockOffset + RequiredSize;
 
     return STATUS_SUCCESS;
 }
 
-NTSTATUS
-GenerateSessionInstance(UCHAR *Buffer,
-                    ULONG BufferSize,
-                    XENIFACE_FDO *fdoData,
-                    ULONG_PTR *byteswritten) {
-    WNODE_SINGLE_INSTANCE *node;
-    ULONG RequiredSize;
-    UCHAR *dbo;
-    UCHAR *InstStr;
-    UNICODE_STRING instance;
-    ULONG* id;
-    XenStoreSession *session;
-    UCHAR *sesbuf;
+static NTSTATUS
+GenerateSessionInstance(
+    IN  PXENIFACE_FDO       Fdo,
+    IN  UCHAR*              Buffer,
+    IN  ULONG               BufferSize,
+    OUT ULONG_PTR*          BytesWritten
+    )
+{
+    WNODE_SINGLE_INSTANCE*  node;
+    ULONG                   RequiredSize;
+    UCHAR*                  dbo;
+    UCHAR*                  InstStr;
+    UNICODE_STRING          instance;
+    ULONG*                  id;
+    PXENSTORE_SESSION       session;
+    UCHAR*                  sesbuf;
+    NTSTATUS                status;
 
+    *BytesWritten = 0;
     if (!AccessWmiBuffer(Buffer, TRUE, &RequiredSize, BufferSize,
-                            WMI_BUFFER, sizeof(WNODE_SINGLE_INSTANCE), &node,
-                            WMI_DONE))
-    {
-        return NodeTooSmall(Buffer, BufferSize, RequiredSize, byteswritten);
-    }
+                         WMI_BUFFER, sizeof(WNODE_SINGLE_INSTANCE), &node,
+                         WMI_DONE))
+        goto fail1;
+
     if (!AccessWmiBuffer(Buffer, TRUE, &RequiredSize, BufferSize,
-                            WMI_BUFFER, sizeof(WNODE_SINGLE_INSTANCE), &node,
-                            WMI_STRINGOFFSET, node->OffsetInstanceName, &InstStr,
-                            WMI_OFFSET, node->DataBlockOffset, 0, &dbo,
-                            WMI_DONE))
-    {
-        return NodeTooSmall(Buffer, BufferSize, RequiredSize, byteswritten);
-    }
+                         WMI_BUFFER, sizeof(WNODE_SINGLE_INSTANCE), &node,
+                         WMI_STRINGOFFSET, node->OffsetInstanceName, &InstStr,
+                         WMI_OFFSET, node->DataBlockOffset, 0, &dbo,
+                         WMI_DONE))
+        goto fail2;
 
     GetCountedUnicodeString(&instance, InstStr);
-    LockSessions(fdoData);
-    if ((session = FindSessionByInstanceLocked(fdoData, &instance))==NULL){
-        UnlockSessions(fdoData);
-        return STATUS_WMI_INSTANCE_NOT_FOUND;
-    }
+
+    AcquireMutex(&Fdo->SessionLock);
+    status = STATUS_WMI_INSTANCE_NOT_FOUND;
+    session = FindSessionByInstanceLocked(Fdo, &instance);
+    if (session == NULL)
+        goto fail3;
 
     if (!AccessWmiBuffer(dbo, FALSE, &RequiredSize, BufferSize-node->DataBlockOffset,
-                            WMI_UINT32, &id,
-                            WMI_STRING,
-                                GetCountedUnicodeStringSize(&session->stringid),
-                                &sesbuf,
-                            WMI_DONE)) {
-        UnlockSessions(fdoData);
-        return NodeTooSmall(Buffer, BufferSize, RequiredSize+node->DataBlockOffset,
-                            byteswritten);
-    }
+                         WMI_UINT32, &id,
+                         WMI_STRING, GetCountedUnicodeStringSize(&session->StringId), &sesbuf,
+                         WMI_DONE))
+        goto fail4;
+
+    *id = session->SessionId;
+    WriteCountedUnicodeString(&session->StringId, sesbuf);
+    ReleaseMutex(&Fdo->SessionLock);
 
-    *id = session->id;
-    WriteCountedUnicodeString(&session->stringid, sesbuf);
-    UnlockSessions(fdoData);
     node->SizeDataBlock = RequiredSize;
     node->WnodeHeader.BufferSize = node->DataBlockOffset + RequiredSize;
-    *byteswritten = node->DataBlockOffset + RequiredSize;
 
+    *BytesWritten = node->DataBlockOffset + RequiredSize;
+
+    return STATUS_SUCCESS;
 
+fail4:
+    ReleaseMutex(&Fdo->SessionLock);
+    return NodeTooSmall(Buffer, BufferSize, RequiredSize + node->DataBlockOffset, BytesWritten);
 
+fail3:
+    ReleaseMutex(&Fdo->SessionLock);
+    return status;
 
-    return STATUS_SUCCESS;
+fail2:
+fail1:
+    return NodeTooSmall(Buffer, BufferSize, RequiredSize, BytesWritten);
 }
 
-
 NTSTATUS
 WmiQueryAllData(
-    IN PXENIFACE_FDO fdoData,
-    IN PIO_STACK_LOCATION stack,
-    OUT ULONG_PTR *byteswritten
-   )
+    IN  PXENIFACE_FDO       Fdo,
+    IN  PIO_STACK_LOCATION  Stack,
+    OUT ULONG_PTR*          BytesWritten
+    )
 {
+    if (IsEqualGUID(Stack->Parameters.WMI.DataPath,
+                    &OBJECT_GUID(XenStoreBase)))
+        return GenerateBaseBlock(Fdo,
+                                 Stack->Parameters.WMI.Buffer,
+                                 Stack->Parameters.WMI.BufferSize,
+                                 BytesWritten);
+
+    if (IsEqualGUID(Stack->Parameters.WMI.DataPath,
+                    &OBJECT_GUID(XenStoreSession)))
+        return GenerateSessionBlock(Fdo,
+                                    Stack->Parameters.WMI.Buffer,
+                                    Stack->Parameters.WMI.BufferSize,
+                                    BytesWritten);
 
-    if (IsEqualGUID(stack->Parameters.WMI.DataPath,
-                    &OBJECT_GUID(XenStoreBase))) {
-        return GenerateBaseBlock(   fdoData,
-                                    stack->Parameters.WMI.Buffer,
-                                    stack->Parameters.WMI.BufferSize,
-                                    byteswritten);
-    }
-    else if (IsEqualGUID(stack->Parameters.WMI.DataPath,
-                         &OBJECT_GUID(XenStoreSession))) {
-        return GenerateSessionBlock(stack->Parameters.WMI.Buffer,
-                                    stack->Parameters.WMI.BufferSize,
-                                    fdoData,
-                                    byteswritten);
-    }
-    else
-        return STATUS_NOT_SUPPORTED;
-
-
+    return STATUS_NOT_SUPPORTED;
 }
 
 NTSTATUS
 WmiQuerySingleInstance(
-    IN PXENIFACE_FDO fdoData,
-    IN PIO_STACK_LOCATION stack,
-    OUT ULONG_PTR *byteswritten
+    IN  PXENIFACE_FDO       Fdo,
+    IN  PIO_STACK_LOCATION  Stack,
+    OUT ULONG_PTR*          BytesWritten
     )
 {
-    if (IsEqualGUID(stack->Parameters.WMI.DataPath,
-                    &OBJECT_GUID(XenStoreBase))) {
-        return GenerateBaseInstance(fdoData,
-                                    stack->Parameters.WMI.Buffer,
-                                    stack->Parameters.WMI.BufferSize,
-                                    byteswritten);
-    }
-    else if (IsEqualGUID(stack->Parameters.WMI.DataPath,
-                         &OBJECT_GUID(XenStoreSession))) {
-        return GenerateSessionInstance(stack->Parameters.WMI.Buffer,
-                                    stack->Parameters.WMI.BufferSize,
-                                    fdoData,
-                                    byteswritten);
-    }
-    else
-        return STATUS_NOT_SUPPORTED;
+    if (IsEqualGUID(Stack->Parameters.WMI.DataPath,
+                    &OBJECT_GUID(XenStoreBase)))
+        return GenerateBaseInstance(Fdo,
+                                    Stack->Parameters.WMI.Buffer,
+                                    Stack->Parameters.WMI.BufferSize,
+                                    BytesWritten);
+
+    if (IsEqualGUID(Stack->Parameters.WMI.DataPath,
+                    &OBJECT_GUID(XenStoreSession)))
+        return GenerateSessionInstance(Fdo,
+                                       Stack->Parameters.WMI.Buffer,
+                                       Stack->Parameters.WMI.BufferSize,
+                                       BytesWritten);
 
+    return STATUS_NOT_SUPPORTED;
 }
 
-
 NTSTATUS
 WmiRegInfo(
-    IN PXENIFACE_FDO fdoData,
-    IN PIO_STACK_LOCATION stack,
-    OUT ULONG_PTR *byteswritten
+    IN  PXENIFACE_FDO       Fdo,
+    IN  PIO_STACK_LOCATION  Stack,
+    OUT ULONG_PTR*          BytesWritten
    )
 {
-    WMIREGINFO *reginfo;
-    WMIREGGUID *guiddata;
-    UCHAR *mofnameptr;
-    UCHAR *regpath;
-    ULONG RequiredSize;
-    int entries = 4;
+    size_t                  mofnamesz;
+    WMIREGGUID*             guid;
+    WMIREGINFO*             reginfo;
+    WMIREGGUID*             guiddata;
+    UCHAR*                  mofnameptr;
+    UCHAR*                  regpath;
+    ULONG                   RequiredSize;
+
+    const int entries = 4;
     const static UNICODE_STRING mofname = RTL_CONSTANT_STRING(L"XENIFACEMOF");
 
-    size_t mofnamesz;
-
-
-    WMIREGGUID * guid;
     Trace("%s\n",__FUNCTION__);
 
-    if  (stack->Parameters.WMI.DataPath == WMIREGISTER) {
+    if  (Stack->Parameters.WMI.DataPath == WMIREGISTER)
         mofnamesz = mofname.Length + sizeof(USHORT);
-    }
-    else {
+    else
         mofnamesz = 0;
-    }
-    if(!AccessWmiBuffer(stack->Parameters.WMI.Buffer, FALSE,
-                        &RequiredSize,
-                        stack->Parameters.WMI.BufferSize,
-                        WMI_BUFFER, sizeof(WMIREGINFO), (UCHAR **)&reginfo,
-                        WMI_BUFFER, entries * sizeof(WMIREGGUID), (UCHAR **)&guiddata,
-                        WMI_STRING, mofnamesz, &mofnameptr,
-                        WMI_STRING, DriverParameters.RegistryPath.Length+sizeof(USHORT),
-                                    &regpath,
-                        WMI_DONE)){
+
+    if (!AccessWmiBuffer(Stack->Parameters.WMI.Buffer, FALSE,
+                         &RequiredSize,
+                         Stack->Parameters.WMI.BufferSize,
+                         WMI_BUFFER, sizeof(WMIREGINFO), (UCHAR **)&reginfo,
+                         WMI_BUFFER, entries * sizeof(WMIREGGUID), (UCHAR **)&guiddata,
+                         WMI_STRING, mofnamesz, &mofnameptr,
+                         WMI_STRING, DriverParameters.RegistryPath.Length + sizeof(USHORT), &regpath,
+                         WMI_DONE)) {
         reginfo->BufferSize = RequiredSize;
-        *byteswritten = sizeof(ULONG);
+        *BytesWritten = sizeof(ULONG);
         return STATUS_BUFFER_TOO_SMALL;
-
     }
-    if (stack->Parameters.WMI.DataPath == WMIREGISTER) {
+
+    if (Stack->Parameters.WMI.DataPath == WMIREGISTER) {
         reginfo->MofResourceName = (ULONG)((ULONG_PTR)mofnameptr - (ULONG_PTR)reginfo);
         WriteCountedUnicodeString(&mofname, mofnameptr);
         reginfo->RegistryPath = (ULONG)((ULONG_PTR)regpath - (ULONG_PTR)reginfo);
@@ -3019,44 +2988,42 @@ WmiRegInfo(
     guid->InstanceCount = 1;
     guid->Guid = OBJECT_GUID(XenStoreBase);
     guid->Flags = WMIREG_FLAG_INSTANCE_PDO;
-    guid->Pdo = (ULONG_PTR)fdoData->PhysicalDeviceObject;
-    ObReferenceObject(fdoData->PhysicalDeviceObject);
+    guid->Pdo = (ULONG_PTR)Fdo->PhysicalDeviceObject;
+    ObReferenceObject(Fdo->PhysicalDeviceObject);
 
     guid = &reginfo->WmiRegGuid[1];
     guid->Guid = OBJECT_GUID(XenStoreSession);
-    guid->Flags =0;
+    guid->Flags = 0;
 
     guid = &reginfo->WmiRegGuid[2];
     guid->InstanceCount = 1;
     guid->Guid = OBJECT_GUID(XenStoreWatchEvent);
     guid->Flags = WMIREG_FLAG_INSTANCE_PDO |
-                WMIREG_FLAG_EVENT_ONLY_GUID ;
-    guid->Pdo = (ULONG_PTR)fdoData->PhysicalDeviceObject;
-    ObReferenceObject(fdoData->PhysicalDeviceObject);
+                  WMIREG_FLAG_EVENT_ONLY_GUID ;
+    guid->Pdo = (ULONG_PTR)Fdo->PhysicalDeviceObject;
+    ObReferenceObject(Fdo->PhysicalDeviceObject);
 
     guid = &reginfo->WmiRegGuid[3];
     guid->InstanceCount = 1;
     guid->Guid = OBJECT_GUID(XenStoreUnsuspendedEvent);
     guid->Flags = WMIREG_FLAG_INSTANCE_PDO |
-                WMIREG_FLAG_EVENT_ONLY_GUID ;
-    guid->Pdo = (ULONG_PTR)fdoData->PhysicalDeviceObject;
-    ObReferenceObject(fdoData->PhysicalDeviceObject);
+                  WMIREG_FLAG_EVENT_ONLY_GUID ;
+    guid->Pdo = (ULONG_PTR)Fdo->PhysicalDeviceObject;
+    ObReferenceObject(Fdo->PhysicalDeviceObject);
 
-
-    *byteswritten = RequiredSize;
+    *BytesWritten = RequiredSize;
     return STATUS_SUCCESS;
 }
 
 NTSTATUS
 WmiRegInfoEx(
-    IN PXENIFACE_FDO fdoData,
-    IN PIO_STACK_LOCATION stack,
-    OUT ULONG_PTR *byteswritten
-   )
+    IN  PXENIFACE_FDO       Fdo,
+    IN  PIO_STACK_LOCATION  Stack,
+    OUT ULONG_PTR*          BytesWritten
+    )
 {
-
     Trace("%s\n",__FUNCTION__);
-    return WmiRegInfo(fdoData, stack, byteswritten);
+    return WmiRegInfo(Fdo, Stack, BytesWritten);
 }
 
 NTSTATUS
@@ -3081,18 +3048,6 @@ WmiProcessMinorFunction(
     }
 
     switch (Stack->MinorFunction) {
-    case IRP_MN_CHANGE_SINGLE_INSTANCE:
-        return WmiChangeSingleInstance(Fdo, Stack);
-    case IRP_MN_CHANGE_SINGLE_ITEM:
-        return WmiChangeSingleItem(Fdo, Stack);
-    case IRP_MN_DISABLE_COLLECTION:
-        return WmiDisableCollection(Fdo, Stack);
-    case IRP_MN_DISABLE_EVENTS:
-        return WmiDisableEvents(Fdo, Stack);
-    case IRP_MN_ENABLE_COLLECTION:
-        return WmiEnableCollection(Fdo, Stack);
-    case IRP_MN_ENABLE_EVENTS:
-        return WmiEnableEvents(Fdo, Stack);
     case IRP_MN_EXECUTE_METHOD:
         return WmiExecuteMethod(Fdo, Stack,  &Irp->IoStatus.Information);
     case IRP_MN_QUERY_ALL_DATA:
@@ -3108,41 +3063,106 @@ WmiProcessMinorFunction(
     }
 }
 
-PCHAR
-WMIMinorFunctionString (
-    __in UCHAR MinorFunction
-)
-/*++
-
-Updated Routine Description:
-    WMIMinorFunctionString does not change in this stage of the function driver.
---*/
-{
-    switch (MinorFunction)
-    {
-    case IRP_MN_CHANGE_SINGLE_INSTANCE:
-        return "IRP_MN_CHANGE_SINGLE_INSTANCE";
-    case IRP_MN_CHANGE_SINGLE_ITEM:
-        return "IRP_MN_CHANGE_SINGLE_ITEM";
-    case IRP_MN_DISABLE_COLLECTION:
-        return "IRP_MN_DISABLE_COLLECTION";
-    case IRP_MN_DISABLE_EVENTS:
-        return "IRP_MN_DISABLE_EVENTS";
-    case IRP_MN_ENABLE_COLLECTION:
-        return "IRP_MN_ENABLE_COLLECTION";
-    case IRP_MN_ENABLE_EVENTS:
-        return "IRP_MN_ENABLE_EVENTS";
-    case IRP_MN_EXECUTE_METHOD:
-        return "IRP_MN_EXECUTE_METHOD";
-    case IRP_MN_QUERY_ALL_DATA:
-        return "IRP_MN_QUERY_ALL_DATA";
-    case IRP_MN_QUERY_SINGLE_INSTANCE:
-        return "IRP_MN_QUERY_SINGLE_INSTANCE";
-    case IRP_MN_REGINFO:
-        return "IRP_MN_REGINFO";
-    default:
-        return "unknown_syscontrol_irp";
+VOID
+WmiFireSuspendEvent(
+    IN  PXENIFACE_FDO   Fdo
+    )
+{
+    Info("Ready to unsuspend Event\n");
+    KeSetEvent(&Fdo->registryWriteEvent, IO_NO_INCREMENT, FALSE);
+
+    if (!Fdo->WmiReady)
+        return;
+
+    Trace("Fire Suspend Event\n");
+    WmiFireEvent(Fdo->Dx->DeviceObject,
+                 (LPGUID)&OBJECT_GUID(XenStoreUnsuspendedEvent),
+                 0,
+                 0,
+                 NULL);
+}
+
+VOID
+WmiSessionsSuspendAll(
+    IN  PXENIFACE_FDO   Fdo
+    )
+{
+    PLIST_ENTRY         ListEntry;
+    PXENSTORE_SESSION   Session;
+
+    AcquireMutex(&Fdo->SessionLock);
+    Trace("Suspend all sessions\n");
+    for (ListEntry = Fdo->SessionHead.Flink;
+         ListEntry != &Fdo->SessionHead;
+         ListEntry = ListEntry->Flink) {
+        Session = CONTAINING_RECORD(ListEntry, XENSTORE_SESSION, ListEntry);
+
+        SessionsSuspendLocked(Fdo, Session);
+    }
+    ReleaseMutex(&Fdo->SessionLock);
+}
+
+VOID
+WmiSessionsResumeAll(
+    IN  PXENIFACE_FDO   Fdo
+    )
+{
+    PLIST_ENTRY         ListEntry;
+    PXENSTORE_SESSION   Session;
+
+    AcquireMutex(&Fdo->SessionLock);
+    Trace("Resume all sessions\n");
+    for (ListEntry = Fdo->SessionHead.Flink;
+         ListEntry != &Fdo->SessionHead;
+         ListEntry = ListEntry->Flink) {
+        Session = CONTAINING_RECORD(ListEntry, XENSTORE_SESSION, ListEntry);
+
+        SessionResumeLocked(Session);
     }
+    ReleaseMutex(&Fdo->SessionLock);
+}
+
+NTSTATUS
+WmiRegister(
+    IN  PXENIFACE_FDO   Fdo
+    )
+{
+    NTSTATUS            status;
+
+    if (Fdo->WmiReady)
+        return STATUS_SUCCESS;
+
+    Trace("%s\n",__FUNCTION__);
+    Info("DRV: XenIface WMI Initialisation\n");
+
+    status = IoWMIRegistrationControl(Fdo->Dx->DeviceObject,
+                                      WMIREG_ACTION_REGISTER);
+    if (!NT_SUCCESS(status))
+        goto fail1;
+
+    Fdo->WmiReady = 1;
+    return STATUS_SUCCESS;
+
+fail1:
+    Error("fail1 (%08x)\n", status);
+    return status;
+}
+
+VOID
+WmiDeregister(
+    IN  PXENIFACE_FDO   Fdo
+    )
+{
+    if (!Fdo->WmiReady)
+        return;
+
+    Info("DRV: XenIface WMI Finalisation\n");
+    Trace("%s\n",__FUNCTION__);
+
+    SessionsRemoveAll(Fdo);
+    (VOID) IoWMIRegistrationControl(Fdo->Dx->DeviceObject,
+                                    WMIREG_ACTION_DEREGISTER);
+    Fdo->WmiReady = 0;
 }
 
 NTSTATUS
-- 
2.32.0.windows.1



From win-pv-devel-bounces@lists.xenproject.org Wed Mar 30 09:39:47 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Wed, 30 Mar 2022 09:39:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.296145.504073 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nZUnr-0002tO-7R; Wed, 30 Mar 2022 09:39:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 296145.504073; Wed, 30 Mar 2022 09:39:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nZUnr-0002tH-4g; Wed, 30 Mar 2022 09:39:47 +0000
Received: by outflank-mailman (input) for mailman id 296145;
 Wed, 30 Mar 2022 09:39:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rAW0=UJ=amazon.co.uk=prvs=0818c4f33=pdurrant@srs-se1.protection.inumbo.net>)
 id 1nZUno-0002sw-Ut
 for win-pv-devel@lists.xenproject.org; Wed, 30 Mar 2022 09:39:45 +0000
Received: from smtp-fw-6001.amazon.com (smtp-fw-6001.amazon.com [52.95.48.154])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 538e9b49-b00d-11ec-8fbc-03012f2f19d4;
 Wed, 30 Mar 2022 11:39:43 +0200 (CEST)
Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO
 email-inbound-relay-pdx-2a-d14a57da.us-west-2.amazon.com) ([10.43.8.6])
 by smtp-border-fw-6001.iad6.amazon.com with ESMTP; 30 Mar 2022 09:39:32 +0000
Received: from EX13D32EUC004.ant.amazon.com
 (pdx1-ws-svc-p6-lb9-vlan2.pdx.amazon.com [10.236.137.194])
 by email-inbound-relay-pdx-2a-d14a57da.us-west-2.amazon.com (Postfix) with
 ESMTPS id 4B7951200BF; Wed, 30 Mar 2022 09:39:31 +0000 (UTC)
Received: from EX13D32EUC003.ant.amazon.com (10.43.164.24) by
 EX13D32EUC004.ant.amazon.com (10.43.164.121) with Microsoft SMTP Server (TLS)
 id 15.0.1497.32; Wed, 30 Mar 2022 09:39:29 +0000
Received: from EX13D32EUC003.ant.amazon.com ([10.43.164.24]) by
 EX13D32EUC003.ant.amazon.com ([10.43.164.24]) with mapi id 15.00.1497.033;
 Wed, 30 Mar 2022 09:39:30 +0000
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 538e9b49-b00d-11ec-8fbc-03012f2f19d4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.co.uk; i=@amazon.co.uk; q=dns/txt;
  s=amazon201209; t=1648633184; x=1680169184;
  h=from:to:date:message-id:references:in-reply-to:
   content-transfer-encoding:mime-version:subject;
  bh=f6m/OwcjrUQPHTRCFOC3D+sH4VKA5k6imq+8OhGskdI=;
  b=q8Cfil8rg7shVwgNMIPVdokvpOlRQHhmlLKQYKXsVo4pGqAH5wq10d6k
   ncMRzw1OO6CkO/4jeUF/TQ3NIvsu2irN5KMTnvQUFfrbjhexQo+fJQ5mY
   ZbrPstz4DOyzoZwnda3+fd4z5jHlQyykIXsK7TszqV6C9tEKqziTGV9o5
   w=;
X-IronPort-AV: E=Sophos;i="5.90,222,1643673600"; 
   d="scan'208";a="190282905"
Subject: RE: [PATCH] Refactor Wmi.c
Thread-Topic: [PATCH] Refactor Wmi.c
From: "Durrant, Paul" <pdurrant@amazon.co.uk>
To: Owen Smith <owen.smith@citrix.com>, "win-pv-devel@lists.xenproject.org"
	<win-pv-devel@lists.xenproject.org>
Thread-Index: AQHYRBku9MX6m58a6EGghnHELfZC8KzXq4mg
Date: Wed, 30 Mar 2022 09:39:30 +0000
Message-ID: <6b30173884a24369a061a9d9ba6bb595@EX13D32EUC003.ant.amazon.com>
References: <20220330093223.659-1-owen.smith@citrix.com>
In-Reply-To: <20220330093223.659-1-owen.smith@citrix.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-transport-fromentityheader: Hosted
x-originating-ip: [10.43.165.112]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

> -----Original Message-----
> From: win-pv-devel <win-pv-devel-bounces@lists.xenproject.org> On Behalf =
Of Owen Smith
> Sent: 30 March 2022 10:32
> To: win-pv-devel@lists.xenproject.org
> Cc: Owen Smith <owen.smith@citrix.com>
> Subject: [EXTERNAL] [PATCH] Refactor Wmi.c
>=20
> CAUTION: This email originated from outside of the organization. Do not c=
lick links or open
> attachments unless you can confirm the sender and know the content is saf=
e.
>=20
>=20
>=20
> * Moves functions to be in related locations in file
> * Formats code to appropriate code style
> * Inlines some functions that are only called from 1 location
> * Uses LIST_ENTRY macros to access linked lists
>=20
> Signed-off-by: Owen Smith <owen.smith@citrix.com>
> ---
>  src/xeniface/wmi.c | 3794 ++++++++++++++++++++++----------------------
>  1 file changed, 1907 insertions(+), 1887 deletions(-)

Eek. 3794 lines of change! That's almost impossible to review. Is there any=
 chance you can break this down?

  Paul


From win-pv-devel-bounces@lists.xenproject.org Wed Mar 30 10:07:41 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Wed, 30 Mar 2022 10:07:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.296180.504122 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nZVEo-00086i-9u; Wed, 30 Mar 2022 10:07:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 296180.504122; Wed, 30 Mar 2022 10:07:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nZVEo-00086b-73; Wed, 30 Mar 2022 10:07:38 +0000
Received: by outflank-mailman (input) for mailman id 296180;
 Wed, 30 Mar 2022 10:07:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xR6P=UJ=citrix.com=prvs=0814e68e9=owen.smith@srs-se1.protection.inumbo.net>)
 id 1nZVEm-00086V-Ge
 for win-pv-devel@lists.xenproject.org; Wed, 30 Mar 2022 10:07:36 +0000
Received: from esa4.hc3370-68.iphmx.com (esa4.hc3370-68.iphmx.com
 [216.71.155.144]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 36e02ffc-b011-11ec-8fbc-03012f2f19d4;
 Wed, 30 Mar 2022 12:07:34 +0200 (CEST)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 36e02ffc-b011-11ec-8fbc-03012f2f19d4
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1648634853;
  h=from:to:subject:date:message-id:references:in-reply-to:
   content-transfer-encoding:mime-version;
  bh=k93P4lbm5Sb8Dn79G7oxAiB82TckNyZ58V3ejNn6PL8=;
  b=Ulliu7RfI+scKe79cQIsxS+rMftOKBlC+AP6dsVM3O2f5q2kn5JtV5Ft
   kwtqvw2h2WuIDXwqYDwGyz5jXbEzanKEFbJD+ayMwrfVrVWktctFnaCt4
   4he0rMrIybd8dBL2SKTYM/0yZvHFsBuGO88JTjOo48ieogfmY5EVHUbzZ
   k=;
Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
X-SBRS: 5.1
X-MesageID: 69909511
X-Ironport-Server: esa4.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:QVlRlqozmSf66K2d0Ggk88iDQWpeBmIQZRIvgKrLsJaIsI4StFCzt
 garIBmDO/yINjP0LYt2YYq39RhTvMfXnINhGgZtq3hhHylE85uZCYyVIHmrMnLJJKUvbq7GA
 +byyDXkBJppJpMJjk71atANlVEliefQAOCU5NfsYkidfyc9IMsaoU8lyrZRbrJA24DjWVvW4
 ouq+aUzBXf+s9JKGjNMg068gEsHUMTa4Fv0aXRnOJinFHeH/5UkJMp3yZOZdhMUcaENdgKOf
 M7RzanRw4/s10xF5uVJMFrMWhZirrb6ZWBig5fNMkSoqkAqSicais7XOBeAAKv+Zvrgc91Zk
 b1wWZKMpQgBLvWUlN0zCzBkCTBdHIIb/I74AH/giJnGp6HGWyOEL/RGCUg3OcsT+/ptAHEI/
 vsdQNwPRknd3aTsmuv9E7QywJR4RCXoFNp3VnVIyi7UC7A9RJHfQ43B5MNC3Sd2jcdLdRrbT
 5RIOGI+MU2ZC/FJEggSNLExvMXvvVXQbwdbpQukh/s+6UGGmWSd15CyaYGIK7RmX/59gUee4
 3rYumj0HBweHNie0iaetGKhgPfVmiH2U55UE6e3ntZyjECJ3GUVFDUfUFKppv//gUn4RtE3F
 qAP0nNw9+5orhXtF4SjGU3jyJKZgvICc8oIS+EDxlmk9rDFwCe2W2w1cR9GN+Vz4afaWgcW/
 lOOmtroAxlmv7uUVW+R+9+okN+iBcQGBTRcPHFZFGPp9/Gm+dhu1UyXEr6PBYbv1rXI9SfML
 ydmRcTUr5EaloY12qqy5jgraBr898GSHmbZCug6N19JDz+Vhqb4P+RECnCBtJ6sybp1qHHb5
 hDofODEsYgz4WmlznDlfQn0NOjBCwy5GDPdm0VzOJIq6i6g/XWuFagJvm0ueRcza5xcI2e1C
 KM2he+3zMUJVJdNRfUqC79d9uxwlfSwfTgbfq68giVyjmhZK1bcoXAGib+41GHxikk8+ZzTy
 r/AGftA+U0yUPw9pBLvHr91+eZymkgWmDOCLbimnk/P+efPOxaopUItbQLmghYRt/jf/m04M
 r93aqO39vmoeLCnOnCMq99Kdzjn7xETXPjLliCeTcbaSiJOE2A9Ef7Bh7Qnfo1uhaNOkenUu
 Hq6XydlJJDX3BUr9S3ihqhfVY7S
IronPort-HdrOrdr: A9a23:YVG9PajJJYoLF/D844LgdWm71XBQX2N13DAbv31ZSRFFG/FwyP
 rAoB1L73PJYWgqNU3IwerwQpVoMkmsiKKdhrNhcItKPTOW8ldASbsN0WKM+UyHJ8STzJ8/6U
 4kSdkENDSSNyk3sS+Z2njALz9I+rDum8rI5ds2jU0dNT2CA5sQqTuRYTzrd3GeMTM2YqbRY6
 DsgvavyQDQG0g/X4CePD0oTuLDr9rEmNbNehgdHSMq7wGIkHeB9KP6OwLw5GZQbxp/hZMZtU
 TVmQ3w4auu99uhzAXH6mPV55NK3PP819p4AtCWgMR9EESjtu/oXvUhZ1SxhkF3nAid0idprD
 AKmWZkAy1H0QKQQohym2qo5+Cv6kdo15ao8y7lvZKqm72HeNt9MbsZuWsRSGqm12Mw+N57y6
 5FxGSfqt5eCg7Bhj3045zSWwhtjVfcmwtqrQc/tQ0pbWIlUs4mkWXfxjImLL4QWCbhrIw3Gu
 hnC8/RoP5QbFOBdnjc+m1i2salUHg/FgqPBhFqgL3f7xFG2HRii0cIzs0WmXkNsJo7Vplf/u
 zBdqBljqtHQMMaZb90QO0BXcy0AGrQRg+kChPbHX33UKUcf37doZ/+57s4oOmsZZwT1ZM33I
 /MVVtJ3FRCD34Gyff+qaGj3iq9MFlVBw6du/22z6IJyYHBeA==
X-IronPort-AV: E=Sophos;i="5.90,222,1643691600"; 
   d="scan'208";a="69909511"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=E5dgTgOhHHIjcFgFaTWHjnntCPdZfojHjYX789rwO/rQF1EfOqGpdxI7ChNzYMwjyT0DRDx1nTVtW67QJU2mlMXm8AiKx3YVvgS2bW3Q0XXSZorYzbaNbBWLV9JIv5KOEN7GUprASPwWmkiLqxNKJ0EO0afhKRkfD835aBn5a81lEsUPrXD5k2V4Ck1c5XIt9J1iySqhEwCOjxl8RNLECwuQ7ZKAaTuef62qi20cpx9hLWtQWyGq4bmgr1gLrbuXkOMcVVgu/kHNJetcT9BPi06/0etvlc8go4Twe5EU4F8l/kViW6KpXQqoyX/NylV4ymdMw7VxVxsoDhwo8OxGZQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=pNxCy6LewsGWH/okj4jZNbhHePucaaWH+8bUmBsMdTc=;
 b=jCH4mjj8YvNdfsWMOcdlrnzh9TS9AkcMsv/Kr4YqF0kgrNp4CPG+B416SKtATEdMxcgPZrKaBIqmSIevQcU8qDtUyLkMLKiRuosn2OcJKyWxJ3BT/1OpZxLTFFZSssnTyl6kxwyjRadj/k3gA7NdTDjodq+01opjosGQcEG6bHI6+Bg+W13InYgkv5kr7BDd8OS1w6MPKWSryEilltkS3cVF1Mm8pp7hvd51LDRwfHeUe4bOo3qpKbDHQfKTDZVrP76EVGNVtWozgk3QFNyYnX84zWzHX87PETlyJtUvHUqG3e8ybq8FBnMy9B6h3M7SBFaMGBqIEOrT2OHa6WX+vw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=pNxCy6LewsGWH/okj4jZNbhHePucaaWH+8bUmBsMdTc=;
 b=GSK3TMqYxkxPbZgX6aULq4HBiypW/Hk7ooVIhmFOSldOHSFERRhFQ1IiI11QItNyGQ4sH5FBe3d9GXjJvW6MJLmog1nIA6qbDQaxDX9iRUarOaZNyqx1B3xgOmjJj8LAhiPMpSneZB8yrjbOImRAEQBY2fjLuO+XIHCkZhQKUNY=
From: Owen Smith <owen.smith@citrix.com>
To: "Durrant, Paul" <pdurrant@amazon.co.uk>,
	"win-pv-devel@lists.xenproject.org" <win-pv-devel@lists.xenproject.org>
Subject: RE: [PATCH] Refactor Wmi.c
Thread-Topic: [PATCH] Refactor Wmi.c
Thread-Index: AQHYRBktznavb3WZD0a/Bmx10iJtaKzXq+kAgAAHLmA=
Date: Wed, 30 Mar 2022 10:07:28 +0000
Message-ID: <DS7PR03MB54774C0D7D550C8BC5A552EEFE1F9@DS7PR03MB5477.namprd03.prod.outlook.com>
References: <20220330093223.659-1-owen.smith@citrix.com>
 <6b30173884a24369a061a9d9ba6bb595@EX13D32EUC003.ant.amazon.com>
In-Reply-To: <6b30173884a24369a061a9d9ba6bb595@EX13D32EUC003.ant.amazon.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 15f9cc20-029e-4179-c825-08da12351924
x-ms-traffictypediagnostic: MWHPR03MB3327:EE_
x-microsoft-antispam-prvs: <MWHPR03MB33276BFD47BD335818EE37D5FE1F9@MWHPR03MB3327.namprd03.prod.outlook.com>
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: poLXjSzU9UJzsaU1zOZO6rRaAgu9nDqRhjNJn7a3+DuC5T4WMxYzwXh8rL05WWKPCaDu5G0bH5Id2BHcNpDE0byYbphi1d4OqP3THfsdxML+uZnRnpQ0ftGGphdfBSxCuU2PjkYHUNvnoZKR7sh1+VgdmYROna/TBwB4l909pgb82d2wkMs/7NWagMPAoVb24RmN5YGa36dVSqX8oOxs5MCKrRZ7zs8rTjyhQdxS2G1AdsXk9wdNXCdvUcfqJb2pPT33nrAK26OaPOs0Mto3DCoPwE3qcnxyJD0twJkRUzl0uvUUbLSUJCBWOTrYnMguWclsQqNREdCJ11YR+LOIlOkGNJ1ChXtv6Uuh8/h7WIuKIdjjwtC5F3WLN+ITAYLsvvL0jOf6W5Rm4WANnSPompoLIOzMVxkXUzjlQHMZ4GTKobKslWvL+EkmxskanmtagEyq4kSVFBwY6YIuo+yLhvOlGz+AWfWfrfbbYpKKU2uFhQjWmdxaSAO7CveIJVbA0duKl6T1wOLFh5NmqhdhLtGfpHOWHnJ3hPi3qiqOFdlNQBDRyFegvNgxpdJZKxJ+zV7968HTvthRuo3N/P50nfVYc9SxWkt/zSoc0ZF4J6X7N5ExRwQtDRs3vM+SwNrJRDIRtvBlptSHJM2kW6mO27s/ixe17qIICzRcfX8hKc6aygdIdSTS9oIs90kfT1neb1g1/oRW00BrIDhVZ/If3w==
x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5477.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(66556008)(66476007)(8676002)(2906002)(26005)(186003)(76116006)(122000001)(64756008)(66946007)(71200400001)(82960400001)(66446008)(9686003)(86362001)(83380400001)(53546011)(508600001)(5660300002)(110136005)(7696005)(316002)(55016003)(8936002)(6506007)(38070700005)(33656002)(38100700002)(44832011)(52536014);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?i7BJSSDP0BgbR8P+5GWYShQqBQZvhVA9ntzsfcPtVZ3pala2i+PYw5FDbwnw?=
 =?us-ascii?Q?ub6HUfbUk0qJFKhqt7fgY/XJ4FjLqhS34xmG3imz8r5JP32e/MwvS/TIYls+?=
 =?us-ascii?Q?uyFK2Cu0NNCL/OCzNcCvZeitlyg09wtghBqyUmw4fL879/+4V5e2gEYVlCXT?=
 =?us-ascii?Q?HEsoZ4cLK/oirPT1qbmwdNdZNKTguMPNxP7/EwVqhrDKMB1oji2QGnTtHeUe?=
 =?us-ascii?Q?WKNDuOttgbBurjBPxbGGcOBOq9qO1jz69ijdMvecXmU8GWc+mNIjb7tFnkyx?=
 =?us-ascii?Q?vpStWohUbcofEMS6kr6L4B68S6fKQANzJK7/nSBag7POz0SPsiSblzFIN+kw?=
 =?us-ascii?Q?juwhowg4c3k6PeGeIjdJugmsJFNIBn7WFMf71/O4hYYkukHgPxPDyymsnadh?=
 =?us-ascii?Q?HpIq4rUNMBFu1v7viF5xpAmNN6mqtjPhSq4zVMsnVgHLNfs9W629g2+v0Dlb?=
 =?us-ascii?Q?avsPP+9VQOr0HHyoV4d08W7vWV8nl1hBAkI/zF8hPpXB4R0Tl1LyFl2REkKg?=
 =?us-ascii?Q?MdJeun4dli3XyJqTHVSk31x+KmIShxxE1ZnJLO+qb/8K0olvlxObVePbnSik?=
 =?us-ascii?Q?zvlIad1wLm9ug5lIvBalck79nOM91DFWsyIdD1U3uR/r07esaPNUMkKv7Gdi?=
 =?us-ascii?Q?G9WGEV+YbKKzV5mKuV8I11fnn1lfoLBBvk38//Djopdq1WmBDfTvj7C+IG7+?=
 =?us-ascii?Q?s8Bdo4mlU3+h9dFxNpwqo/cETYYjuGjWnfDs/PQhUKQ4oSABsLrnmJg7ZQqV?=
 =?us-ascii?Q?PZJad8OjRq23vqP6XXNlCeXOJa/3AxKmgLYMzZXEXvo8e2fAttS8n9nA7G75?=
 =?us-ascii?Q?7XotJX/7IS1qisi17UOggYN9JQI5MJ7Osw64RCgCNRf+mJbqFHZFHklM9cz/?=
 =?us-ascii?Q?7/1AjEtyuB2X1AGwuXwkD5SePluVdXV8mfGkM29OlVrn2ABV2DBmBzjc07G4?=
 =?us-ascii?Q?RoByjSTyLeTDRO3owf3AMDtBySQQdKVIi/LfXBsPC+RR+VFaQf4Dr2oWNf0Y?=
 =?us-ascii?Q?gvEXSxakD7s0zo/4zg5iaCPRg3k//fqTyeqmGMXRuFW2Pp6YA5JqIKwEh3HC?=
 =?us-ascii?Q?fAl8kKSds27/IP3K8DLq+jlArI1Rs1tf195VookC3fJlni0Xa1C2CuThd/ei?=
 =?us-ascii?Q?h7r6vEnzgdGft7gH3s+k02SwqOHCMuQa/FUFlSCl0BadsYqom0ofqt5LTkcV?=
 =?us-ascii?Q?lseY9chrh4wA4oXhII8crz3CXPn6a0+fMzL9kKO2TYJHGWis332SIQlBt0QF?=
 =?us-ascii?Q?x1SuIo3Qodnmuh2SfWqYslXz6QJvfkcc66mCh6E6N4/7MW8XQCFyTeorWnUm?=
 =?us-ascii?Q?kvnZ6K0DbLDtFBGmvN+xgvGqGtZGPxWHzBHdS+LVKN08kPXey5iRDmVc7s1T?=
 =?us-ascii?Q?dYH+JBrqCMtUnGx2WNwi4xHIp5FpshrK1ERn/c33NKw6ogmXMbMf3NcB2cms?=
 =?us-ascii?Q?EYmEU1Vd3ugTMtRH6qOz1hrMVjVuvfeZv50hNx4buDdJIldjU23lVfN9ySpV?=
 =?us-ascii?Q?d8uhccJgo6HPpEBhXYQd4i0Jq+l+55tBuuULKzuqXYs0hSmmOmRRZolYNZLU?=
 =?us-ascii?Q?pP+deVKkaCNRO45VLRI1ewWpYOIbEYNWITh14IY2XBcvk8IIer9Cga6R6+O/?=
 =?us-ascii?Q?9nwK8kBovkSVBbhQfS+PyZ41/10YoqXuZZOMJ2AfF0rCmKwpZA+0nI6bgmnv?=
 =?us-ascii?Q?1M8cOES/ZkiT8sJ1601Fv+TsTFIjO3PAD+qKx9CHdpzMrE23SClsJEdGFWga?=
 =?us-ascii?Q?3uRz42un5w=3D=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5477.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 15f9cc20-029e-4179-c825-08da12351924
X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Mar 2022 10:07:29.0129
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: A6xMTfA+yjgkNG0ld2lUiVurIx08PZBGI3nFU5JVs8BS4P64XEokLEYLsM1OqT00a9auZBAs/R9RXQvdauDGXA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR03MB3327
X-OriginatorOrg: citrix.com

-----Original Message-----
From: Durrant, Paul <pdurrant@amazon.co.uk>=20
Sent: 30 March 2022 10:40
To: Owen Smith <owen.smith@citrix.com>; win-pv-devel@lists.xenproject.org
Subject: RE: [PATCH] Refactor Wmi.c

[CAUTION - EXTERNAL EMAIL] DO NOT reply, click links, or open attachments u=
nless you have verified the sender and know the content is safe.

> -----Original Message-----
> From: win-pv-devel <win-pv-devel-bounces@lists.xenproject.org> On=20
> Behalf Of Owen Smith
> Sent: 30 March 2022 10:32
> To: win-pv-devel@lists.xenproject.org
> Cc: Owen Smith <owen.smith@citrix.com>
> Subject: [EXTERNAL] [PATCH] Refactor Wmi.c
>=20
> CAUTION: This email originated from outside of the organization. Do=20
> not click links or open attachments unless you can confirm the sender and=
 know the content is safe.
>=20
>=20
>=20
> * Moves functions to be in related locations in file
> * Formats code to appropriate code style
> * Inlines some functions that are only called from 1 location
> * Uses LIST_ENTRY macros to access linked lists
>=20
> Signed-off-by: Owen Smith <owen.smith@citrix.com>
> ---
>  src/xeniface/wmi.c | 3794=20
> ++++++++++++++++++++++----------------------
>  1 file changed, 1907 insertions(+), 1887 deletions(-)

Eek. 3794 lines of change! That's almost impossible to review. Is there any=
 chance you can break this down?

  Paul


Its not easy to break down a refactor like this, 1 small change has many kn=
ock on changes throughout the file.
Its probably easier to apply locally and compare side by side, but I'm not =
expecting this to be an easy or quick review - I probably should have poste=
d 1st with RFC

Owen


From win-pv-devel-bounces@lists.xenproject.org Wed Mar 30 10:11:23 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Wed, 30 Mar 2022 10:11:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.296187.504125 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nZVIR-0000Z8-KE; Wed, 30 Mar 2022 10:11:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 296187.504125; Wed, 30 Mar 2022 10:11:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nZVIR-0000Z1-HQ; Wed, 30 Mar 2022 10:11:23 +0000
Received: by outflank-mailman (input) for mailman id 296187;
 Wed, 30 Mar 2022 10:11:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4Soy=UJ=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1nZVIQ-0000Yv-Ls
 for win-pv-devel@lists.xenproject.org; Wed, 30 Mar 2022 10:11:22 +0000
Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com
 [2a00:1450:4864:20::336])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id be7dc732-b011-11ec-a405-831a346695d4;
 Wed, 30 Mar 2022 12:11:21 +0200 (CEST)
Received: by mail-wm1-x336.google.com with SMTP id n35so11897292wms.5
 for <win-pv-devel@lists.xenproject.org>; Wed, 30 Mar 2022 03:11:20 -0700 (PDT)
Received: from [10.7.237.6] (54-240-197-230.amazon.com. [54.240.197.230])
 by smtp.gmail.com with ESMTPSA id
 a18-20020a05600c349200b0038ca453a887sm4559028wmq.19.2022.03.30.03.11.19
 for <win-pv-devel@lists.xenproject.org>
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 30 Mar 2022 03:11:19 -0700 (PDT)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: be7dc732-b011-11ec-a405-831a346695d4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=message-id:date:mime-version:user-agent:reply-to:subject
         :content-language:to:references:from:in-reply-to
         :content-transfer-encoding;
        bh=4erVpblV5nS7yN+dyf2wwsCYQF39vUppMHJe0nnviak=;
        b=WYcqHyd1GQCYKkzaOw4b4uC+TkbJRjWRO7bPFTgDzuS3O1fS4ZlgDJWMM9xx+JgDCe
         pasu5rFpkpx869IF7ZaKezNc1wPZYEZVLdmqiv6U4k9vRdjiYj7c72PU3T4WNRQCg6Z2
         BhBB728NJdGnm9BMiejWxf8AOej8+A6szzEi2kLunMXu/tV6fA6pHcmiO/Jzg42K2wDb
         4bB0pSb93DANm7BKzWoXVVeYiYIgvernHwlqChwR/W11xK1s/PWuktNsLOtknIXtSKPN
         +ex+ypNyTWLwnSBoInHpHwEubtK4kn6Bo4i8VCZ4AXWy5gcfJXzUYuCF1iFJA30KQ7i4
         8+dw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=x-gm-message-state:message-id:date:mime-version:user-agent:reply-to
         :subject:content-language:to:references:from:in-reply-to
         :content-transfer-encoding;
        bh=4erVpblV5nS7yN+dyf2wwsCYQF39vUppMHJe0nnviak=;
        b=pu9sJgv30B5u9CW/HgfPNKAsn2wBKMKxsFuXqrj87ydYfcNE7KYbHTdzpXGdlWhTRs
         B4njIfzjGcua4qvvrvae/T5CH0uoI/wzJGPm9c5t3VXwtd9Kjr92lPb0nKsI9gJ3moWn
         TtuZgkATwDZ7i+Bwr4jCXMivcuXnf8Q9UB7xNczDsyC2qukGu6Qcah7BPRc0PvIxmIsq
         YodJS+PW0kvuloOrFDHI+ACbCHRx/Xyz4iLpcg0Lh1sapH+C4GPbOp2DMIgOp4kg+u1K
         UW+ZZTsYWfjRjVt2JFH4hfXsr7CjV1GW5wwi0oGDZdvd+KXK4JAZ3L+Ui3xBS2CLk8rG
         fTGg==
X-Gm-Message-State: AOAM532aSX+MKHrZJ0jpsF3cuiCddwx5ybuVRDTjLaLijmjXROdYDraj
	hD1OuMtvj3d01hQcqRA3TjzmjrOFFda+Pw==
X-Google-Smtp-Source: ABdhPJxvvVvpk+W3df+lmCngNjsmtzuHGKFuFqtI01kWcp75sCSfwbqKlWv3539Tk20uySX//JiTdw==
X-Received: by 2002:a05:600c:214a:b0:38c:aa5d:1872 with SMTP id v10-20020a05600c214a00b0038caa5d1872mr3714253wml.9.1648635079648;
        Wed, 30 Mar 2022 03:11:19 -0700 (PDT)
Message-ID: <28abd23c-3e72-4a6a-37aa-594a9a130334@gmail.com>
Date: Wed, 30 Mar 2022 11:11:18 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.7.0
Reply-To: paul@xen.org
Subject: Re: [PATCH] Refactor Wmi.c
Content-Language: en-US
To: win-pv-devel@lists.xenproject.org
References: <20220330093223.659-1-owen.smith@citrix.com>
 <6b30173884a24369a061a9d9ba6bb595@EX13D32EUC003.ant.amazon.com>
 <DS7PR03MB54774C0D7D550C8BC5A552EEFE1F9@DS7PR03MB5477.namprd03.prod.outlook.com>
From: "Durrant, Paul" <xadimgnik@gmail.com>
In-Reply-To: <DS7PR03MB54774C0D7D550C8BC5A552EEFE1F9@DS7PR03MB5477.namprd03.prod.outlook.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 30/03/2022 11:07, Owen Smith wrote:
> -----Original Message-----
> From: Durrant, Paul <pdurrant@amazon.co.uk>
> Sent: 30 March 2022 10:40
> To: Owen Smith <owen.smith@citrix.com>; win-pv-devel@lists.xenproject.org
> Subject: RE: [PATCH] Refactor Wmi.c
> 
> [CAUTION - EXTERNAL EMAIL] DO NOT reply, click links, or open attachments unless you have verified the sender and know the content is safe.
> 
>> -----Original Message-----
>> From: win-pv-devel <win-pv-devel-bounces@lists.xenproject.org> On
>> Behalf Of Owen Smith
>> Sent: 30 March 2022 10:32
>> To: win-pv-devel@lists.xenproject.org
>> Cc: Owen Smith <owen.smith@citrix.com>
>> Subject: [EXTERNAL] [PATCH] Refactor Wmi.c
>>
>> CAUTION: This email originated from outside of the organization. Do
>> not click links or open attachments unless you can confirm the sender and know the content is safe.
>>
>>
>>
>> * Moves functions to be in related locations in file
>> * Formats code to appropriate code style
>> * Inlines some functions that are only called from 1 location
>> * Uses LIST_ENTRY macros to access linked lists
>>
>> Signed-off-by: Owen Smith <owen.smith@citrix.com>
>> ---
>>   src/xeniface/wmi.c | 3794
>> ++++++++++++++++++++++----------------------
>>   1 file changed, 1907 insertions(+), 1887 deletions(-)
> 
> Eek. 3794 lines of change! That's almost impossible to review. Is there any chance you can break this down?
> 
>    Paul
> 
> 
> Its not easy to break down a refactor like this, 1 small change has many knock on changes throughout the file.
> Its probably easier to apply locally and compare side by side, but I'm not expecting this to be an easy or quick review - I probably should have posted 1st with RFC
> 

Can you assert that there is no functional change? I will consider 
taking it on trust if so.

   Paul



From win-pv-devel-bounces@lists.xenproject.org Wed Mar 30 14:02:24 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Wed, 30 Mar 2022 14:02:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.296369.504453 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nZYty-00045N-S2; Wed, 30 Mar 2022 14:02:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 296369.504453; Wed, 30 Mar 2022 14:02:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nZYty-00045G-PG; Wed, 30 Mar 2022 14:02:22 +0000
Received: by outflank-mailman (input) for mailman id 296369;
 Wed, 30 Mar 2022 14:02:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xR6P=UJ=citrix.com=prvs=0814e68e9=owen.smith@srs-se1.protection.inumbo.net>)
 id 1nZYtx-00045A-Te
 for win-pv-devel@lists.xenproject.org; Wed, 30 Mar 2022 14:02:22 +0000
Received: from esa5.hc3370-68.iphmx.com (esa5.hc3370-68.iphmx.com
 [216.71.155.168]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 02a40c69-b032-11ec-a405-831a346695d4;
 Wed, 30 Mar 2022 16:02:20 +0200 (CEST)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 02a40c69-b032-11ec-a405-831a346695d4
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1648648939;
  h=from:to:cc:subject:date:message-id:mime-version:
   content-transfer-encoding;
  bh=EkLRBih1/S9MVpf7aWTnUzzKWecFhMiJxTH0byJSYhg=;
  b=Glmd9ubOmIinDdAtHzvpOwq9Put5eKTys/uO7OJ1NHnLLeyQIFojdYTO
   G3WGlLU+o+US9vO8kBx6IIv+gzkL8HEtjGVbcBZfXLOgKTnXxKkQm9pSy
   342ffqjG0MvpHXg7Y5Dprj/VMANRmUIIUk/oXRVo0LnBfHvYYFIb5ANes
   A=;
Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
X-SBRS: 5.1
X-MesageID: 66988876
X-Ironport-Server: esa5.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:a7Y6NaCbK+NejhVW/0Tjw5YqxClBgxIJ4kV8jS/XYbTApDslhmEPy
 mBKDW6Ab/eJMTP3ft0nYYXi8xtX657UnIdhQQY4rX1jcSlH+JHPbTi7wuYcHM8wwunrFh8PA
 xA2M4GYRCwMZiaA4E/raNANlFEkvU2ybuOU5NXsZ2YgHWeIdA970Ug5w7Vi2tYy6TSEK1jlV
 e3a8pW31GCNg1aYAkpMg05UgEoy1BhakGpwUm0WPZinjneH/5UmJMt3yZWKB2n5WuFp8tuSH
 I4v+l0bElTxpH/BAvv9+lryn9ZjrrT6ZWBigVIOM0Sub4QrSoXfHc/XOdJFAXq7hQllkPhbk
 9IVuKy3Yj1xEfXHgd0iaBJ5HCtxaPguFL/veRBTsOSWxkzCNXDt3+9vHAc9OohwFuRfWD8Us
 6ZCcXZUM07F17neLLGTE4GAguwkM8Ttepgau21g5TrYEewnUdbIRKCiCdpwgmlh1pAeQqi2i
 8wxUid1RSrnPjZzFw0MUpgXobz0oCjdWmgNwL6SjfVuuDWCpOBr65DyMdyQYsLPSchLk0Kwo
 mPd43+/EhwcLMaYyzeO7jSrnOCnoM/gcNtMTvvirKcs2QDNgDxIYPELabelidPijRaza9V0E
 RQ7oiE3lJcJ/22kRNaoCnVUv0W4lhIbXtNRFcgz5weM1rfY7m6lO4QUctJSQId47ZFrHFTGw
 nfMxoq0XmI37NV5XFrHrt+pQSWO1T/5xIPoTQsNVkM77tbqu+nfZTqfH484QMZZYjAYcAwcI
 gxmTgBj393/buZRjs1XGGwrZRr2/PAlqSZvu23qspqNtF8RWWJfW6Sm6ELA8dFLJ5uDQ1+Ks
 RAswpbCvL1eV8jXzHbVHI3h+Y1FAd7falUwZnY1QvEcG8mFoSb/Lei8HhkgTKuWDir0UWCwO
 xKC0e+gzJRSIGGrfcdKj3GZUKwXIVzbPY29DJj8N4MWCrAoLVPv1Hw+NCa4gjG2+GBxwP5XB
 HtuWZv1ZV4AF79dxSa7L89El+dDKtYWnjiIG/gWDn2PjNKjWZJiYepcaALVNLFgtstpYmz9q
 r5iCidD8D0HOMWWX8Ud2dV7wYwiRZTjOa3Llg==
IronPort-HdrOrdr: A9a23:Yi6mEqoPF8GEfDH5ahho02UaV5oheYIsimQD101hICG8cqSj+P
 xG+85rsCMc6QxhP03I9urgBEDtex7hHP1OkOss1NWZPDUO0VHAROoJ0WKL+UyFJ8SUzIBgPM
 lbH5SWcOeAbmRHsQ==
X-IronPort-AV: E=Sophos;i="5.90,222,1643691600"; 
   d="scan'208";a="66988876"
From: Owen Smith <owen.smith@citrix.com>
To: <win-pv-devel@lists.xenproject.org>
CC: Owen Smith <owen.smith@citrix.com>
Subject: [PATCH 1/2] Move BlkifRing CacheCreate/Destroy to inline functions
Date: Wed, 30 Mar 2022 15:02:10 +0100
Message-ID: <20220330140211.909-1-owen.smith@citrix.com>
X-Mailer: git-send-email 2.32.0.windows.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

Signed-off-by: Owen Smith <owen.smith@citrix.com>
---
 src/xenvbd/ring.c | 230 +++++++++++++++++++++++++++-------------------
 1 file changed, 135 insertions(+), 95 deletions(-)

diff --git a/src/xenvbd/ring.c b/src/xenvbd/ring.c
index 1ea1250..4482c31 100644
--- a/src/xenvbd/ring.c
+++ b/src/xenvbd/ring.c
@@ -1709,63 +1709,26 @@ RingWatchdog(
     return STATUS_SUCCESS;
 }
 
-static NTSTATUS
-BlkifRingCreate(
-    IN  PXENVBD_RING        Ring,
-    IN  ULONG               Index,
-    OUT PXENVBD_BLKIF_RING* BlkifRing
+static FORCEINLINE NTSTATUS
+BlkifRingCacheCreate(
+    IN  PXENVBD_BLKIF_RING  BlkifRing
     )
 {
+    PXENVBD_RING            Ring;
     PXENVBD_FRONTEND        Frontend;
-    ULONG                   Length;
-    PCHAR                   Path;
     CHAR                    Name[MAX_NAME_LEN];
     NTSTATUS                status;
 
+    Ring = BlkifRing->Ring;
     Frontend = Ring->Frontend;
 
-    Length = (ULONG)strlen(FrontendGetFrontendPath(Frontend)) +
-        (ULONG)strlen("/queue-xxx");
-
-    Path = __RingAllocate(Length + 1);
-
-    status = STATUS_NO_MEMORY;
-    if (Path == NULL)
-        goto fail1;
-
-    status = RtlStringCchPrintfA(Path,
-                                 Length,
-                                 "%s/queue-%u",
-                                 FrontendGetFrontendPath(Frontend),
-                                 Index);
-    if (!NT_SUCCESS(status))
-        goto fail2;
-
-    *BlkifRing = __RingAllocate(sizeof(XENVBD_BLKIF_RING));
-
-    status = STATUS_NO_MEMORY;
-    if (*BlkifRing == NULL)
-        goto fail3;
-
-    (*BlkifRing)->Ring = Ring;
-    (*BlkifRing)->Index = Index;
-    (*BlkifRing)->Path = Path;
-    Path = NULL;
-
-    InitializeListHead(&(*BlkifRing)->SrbQueue);
-    InitializeListHead(&(*BlkifRing)->ShutdownQueue);
-    InitializeListHead(&(*BlkifRing)->SubmittedList);
-    InitializeListHead(&(*BlkifRing)->PreparedQueue);
-
-    KeInitializeThreadedDpc(&(*BlkifRing)->Dpc, BlkifRingDpc, *BlkifRing);
-
     status = RtlStringCbPrintfA(Name,
                                 sizeof(Name),
                                 "vbd_%u_queue_%u_request",
                                 FrontendGetTargetId(Frontend),
-                                Index);
+                                BlkifRing->Index);
     if (!NT_SUCCESS(status))
-        goto fail4;
+        goto fail1;
 
     status = XENBUS_CACHE(Create,
                           &Ring->CacheInterface,
@@ -1777,18 +1740,18 @@ BlkifRingCreate(
                           BlkifRingRequestDtor,
                           BlkifRingAcquireLock,
                           BlkifRingReleaseLock,
-                          *BlkifRing,
-                          &(*BlkifRing)->RequestCache);
+                          BlkifRing,
+                          &BlkifRing->RequestCache);
     if (!NT_SUCCESS(status))
-        goto fail5;
+        goto fail2;
 
     status = RtlStringCbPrintfA(Name,
                                 sizeof(Name),
                                 "vbd_%u_queue_%u_segment",
                                 FrontendGetTargetId(Frontend),
-                                Index);
+                                BlkifRing->Index);
     if (!NT_SUCCESS(status))
-        goto fail6;
+        goto fail3;
 
     status = XENBUS_CACHE(Create,
                           &Ring->CacheInterface,
@@ -1800,18 +1763,18 @@ BlkifRingCreate(
                           BlkifRingSegmentDtor,
                           BlkifRingAcquireLock,
                           BlkifRingReleaseLock,
-                          *BlkifRing,
-                          &(*BlkifRing)->SegmentCache);
+                          BlkifRing,
+                          &BlkifRing->SegmentCache);
     if (!NT_SUCCESS(status))
-        goto fail7;
+        goto fail4;
 
     status = RtlStringCbPrintfA(Name,
                                 sizeof(Name),
                                 "vbd_%u_queue_%u_indirect",
                                 FrontendGetTargetId(Frontend),
-                                Index);
+                                BlkifRing->Index);
     if (!NT_SUCCESS(status))
-        goto fail8;
+        goto fail5;
 
     status = XENBUS_CACHE(Create,
                           &Ring->CacheInterface,
@@ -1823,43 +1786,135 @@ BlkifRingCreate(
                           BlkifRingIndirectDtor,
                           BlkifRingAcquireLock,
                           BlkifRingReleaseLock,
-                          *BlkifRing,
-                          &(*BlkifRing)->IndirectCache);
-    if (!NT_SUCCESS(status))
-        goto fail9;
-
-    status = ThreadCreate(RingWatchdog,
-                          *BlkifRing,
-                          &(*BlkifRing)->WatchdogThread);
+                          BlkifRing,
+                          &BlkifRing->IndirectCache);
     if (!NT_SUCCESS(status))
-        goto fail10;
+        goto fail6;
 
     return STATUS_SUCCESS;
 
-fail10:
-    Error("fail10\n");
+fail6:
+    Error("fail6\n");
+
+fail5:
+    Error("fail5\n");
+
     XENBUS_CACHE(Destroy,
                  &Ring->CacheInterface,
-                 (*BlkifRing)->IndirectCache);
-    (*BlkifRing)->IndirectCache = NULL;
-fail9:
-    Error("fail9\n");
-fail8:
-    Error("fail8\n");
+                 BlkifRing->SegmentCache);
+    BlkifRing->SegmentCache = NULL;
+
+fail4:
+    Error("fail4\n");
+
+fail3:
+    Error("fail3\n");
+
     XENBUS_CACHE(Destroy,
                  &Ring->CacheInterface,
-                 (*BlkifRing)->SegmentCache);
-    (*BlkifRing)->SegmentCache = NULL;
-fail7:
-    Error("fail7\n");
-fail6:
-    Error("fail6\n");
+                 BlkifRing->RequestCache);
+    BlkifRing->RequestCache = NULL;
+
+fail2:
+    Error("fail2\n");
+
+fail1:
+    Error("fail1 %08x\n", status);
+
+    return status;
+}
+
+static FORCEINLINE VOID
+BlkifRingCacheDestroy(
+    IN  PXENVBD_BLKIF_RING  BlkifRing
+    )
+{
+    PXENVBD_RING            Ring;
+
+    Ring = BlkifRing->Ring;
+
     XENBUS_CACHE(Destroy,
                  &Ring->CacheInterface,
-                 (*BlkifRing)->RequestCache);
-    (*BlkifRing)->RequestCache = NULL;
+                 BlkifRing->IndirectCache);
+    BlkifRing->IndirectCache = NULL;
+
+    XENBUS_CACHE(Destroy,
+                 &Ring->CacheInterface,
+                 BlkifRing->SegmentCache);
+    BlkifRing->SegmentCache = NULL;
+
+    XENBUS_CACHE(Destroy,
+                 &Ring->CacheInterface,
+                 BlkifRing->RequestCache);
+    BlkifRing->RequestCache = NULL;
+}
+
+static NTSTATUS
+BlkifRingCreate(
+    IN  PXENVBD_RING        Ring,
+    IN  ULONG               Index,
+    OUT PXENVBD_BLKIF_RING* BlkifRing
+    )
+{
+    PXENVBD_FRONTEND        Frontend;
+    ULONG                   Length;
+    PCHAR                   Path;
+    NTSTATUS                status;
+
+    Frontend = Ring->Frontend;
+
+    Length = (ULONG)strlen(FrontendGetFrontendPath(Frontend)) +
+        (ULONG)strlen("/queue-xxx");
+
+    Path = __RingAllocate(Length + 1);
+
+    status = STATUS_NO_MEMORY;
+    if (Path == NULL)
+        goto fail1;
+
+    status = RtlStringCchPrintfA(Path,
+                                 Length,
+                                 "%s/queue-%u",
+                                 FrontendGetFrontendPath(Frontend),
+                                 Index);
+    if (!NT_SUCCESS(status))
+        goto fail2;
+
+    *BlkifRing = __RingAllocate(sizeof(XENVBD_BLKIF_RING));
+
+    status = STATUS_NO_MEMORY;
+    if (*BlkifRing == NULL)
+        goto fail3;
+
+    (*BlkifRing)->Ring = Ring;
+    (*BlkifRing)->Index = Index;
+    (*BlkifRing)->Path = Path;
+    Path = NULL;
+
+    InitializeListHead(&(*BlkifRing)->SrbQueue);
+    InitializeListHead(&(*BlkifRing)->ShutdownQueue);
+    InitializeListHead(&(*BlkifRing)->SubmittedList);
+    InitializeListHead(&(*BlkifRing)->PreparedQueue);
+
+    KeInitializeThreadedDpc(&(*BlkifRing)->Dpc, BlkifRingDpc, *BlkifRing);
+
+    status = BlkifRingCacheCreate(*BlkifRing);
+    if (!NT_SUCCESS(status))
+        goto fail4;
+
+    status = ThreadCreate(RingWatchdog,
+                          *BlkifRing,
+                          &(*BlkifRing)->WatchdogThread);
+    if (!NT_SUCCESS(status))
+        goto fail5;
+
+    return STATUS_SUCCESS;
+
 fail5:
     Error("fail5\n");
+
+    BlkifRingCacheDestroy(*BlkifRing);
+
 fail4:
     Error("fail4\n");
 
@@ -1893,26 +1948,11 @@ BlkifRingDestroy(
     IN  PXENVBD_BLKIF_RING  BlkifRing
     )
 {
-    PXENVBD_RING            Ring = BlkifRing->Ring;
-
     ThreadAlert(BlkifRing->WatchdogThread);
     ThreadJoin(BlkifRing->WatchdogThread);
     BlkifRing->WatchdogThread = NULL;
 
-    XENBUS_CACHE(Destroy,
-                 &Ring->CacheInterface,
-                 BlkifRing->IndirectCache);
-    BlkifRing->IndirectCache = NULL;
-
-    XENBUS_CACHE(Destroy,
-                 &Ring->CacheInterface,
-                 BlkifRing->SegmentCache);
-    BlkifRing->SegmentCache = NULL;
-
-    XENBUS_CACHE(Destroy,
-                 &Ring->CacheInterface,
-                 BlkifRing->RequestCache);
-    BlkifRing->RequestCache = NULL;
+    BlkifRingCacheDestroy(BlkifRing);
 
     RtlZeroMemory(&BlkifRing->Dpc, sizeof(KDPC));
 
-- 
2.32.0.windows.1



From win-pv-devel-bounces@lists.xenproject.org Wed Mar 30 14:04:26 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Wed, 30 Mar 2022 14:04:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.296373.504458 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nZYvy-0004AL-25; Wed, 30 Mar 2022 14:04:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 296373.504458; Wed, 30 Mar 2022 14:04:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nZYvx-0004AD-VT; Wed, 30 Mar 2022 14:04:25 +0000
Received: by outflank-mailman (input) for mailman id 296373;
 Wed, 30 Mar 2022 14:04:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xR6P=UJ=citrix.com=prvs=0814e68e9=owen.smith@srs-se1.protection.inumbo.net>)
 id 1nZYvw-0004A7-Ee
 for win-pv-devel@lists.xenproject.org; Wed, 30 Mar 2022 14:04:24 +0000
Received: from esa1.hc3370-68.iphmx.com (esa1.hc3370-68.iphmx.com
 [216.71.145.142]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4be981e0-b032-11ec-8fbc-03012f2f19d4;
 Wed, 30 Mar 2022 16:04:22 +0200 (CEST)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4be981e0-b032-11ec-8fbc-03012f2f19d4
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1648649062;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=PBUOiBo8Ar66gFykbsOFoFnX1vkq9LX/lx13Yor/20w=;
  b=CaDPXehKsZjzy8pKwKr68rHmxzzHo2WQ9fFj3LFyu4yto9qzg754JWRK
   fDbGQ5tYgIErtGjxbUIsFHSxsYlDHFarrEpGUA1MyiFraASOqGHzNUFNn
   4Fgt/3aHHygS0pnIG+VnUbi4tmp8jmuM994WjjzFRT56VfmvgQEdfq/xr
   k=;
Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
X-SBRS: 5.1
X-MesageID: 67952087
X-Ironport-Server: esa1.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:UvbglqIRTqKYjUzVFE+RtZUlxSXFcZb7ZxGr2PjKsXjdYENSgWZWy
 jBMWW3UOv+PNjHxLtx2a4/j/U0A7MTRmINmHVFlqX01Q3x08seUXt7xwmUcns+xwm8vaGo9s
 q3yv/GZdJhcokf0/0vrav67xZVF/fngqoDUUYYoAQgsA148IMsdoUg7wbRh2dUw2YLR7z6l4
 rseneWOYDdJ5BYsWo4kw/rrRMRH5amaVJsw5zTSVNgT1LPsvyB94KE3fMldG0DQUIhMdtNWc
 s6YpF2PEsE1yD92Yj+tuu6TnkTn2dc+NyDW4pZdc/DKbhSvOkXee0v0XRYRQR4/ttmHozx+4
 Pds8szuERg0Ao/VpO8XeRZSCTpHHYQTrdcrIVDn2SCS50jPcn+qyPRyFkAme4Yf/46bA0kXq
 6ZecmpUKEne2aTmm9pXScE17ignBM70MYVZoXRh0THxBvc6W5HTBa7N4Le02R9u2ZETR66OO
 KL1bxJFQj+cbCxrGGswN8kwsvya13fzLxNH/Qf9Sa0fvDGIkV0ZPKLWGMXUfJmSW4BZk1iVo
 krC/n/lGVcKOdqH0z2H/3mwwOjVkkvGtJk6TePisKQw2RvKmzJVWEZ+uUaHTeeRqG/5C4wHN
 FEo2DMSpqIY+0axVeDddkjtyJKbhSI0V91VGuw8zQiCzKvI/gqUblQ5oi59hM8O75FvG2Fzv
 rOdt5awXGE07uXJIZ6I3u3MxQ5eLxT5OoPricUsaQIeq+fur4go5v4kZoYySfXl5jEZ9NyZ/
 txrkMTcr+hL5SLo///ilbwiv95KjsKTJuLSzl+KNl9JFisjOOaYi3WAsDA3F8poIoeDVUWmt
 3MZgcWY5+1mJcjTyHzQEL9URunzv6rt3NjgbbhHRcNJG9OFoSPLQGytyGsmeBcB3jgsJ1cFn
 3M/SSsOvcQObRNGnId8Yp6rCtRC8EQTPY+NaxwgVfIXOsIZXFbepElGPBfMt0iwwBlEufxuY
 v+zLJfzZUv2/Iw6lVJasc9GiuR1rs3/rEuOLa3GI+OPjOPGPSXPGO5fWLZMB8hghJ65TMzu2
 443H6O3J993C7ClCsUL2eb/9Ww3EEU=
IronPort-HdrOrdr: A9a23:gd3LWap06gMV6k8I6kNXAA8aV5oteYIsimQD101hICG8cqSj+P
 xG+85rsiMc6QxhIU3I9urgBEDtex7hHNtOkOss1NSZLW3bUQmTTL2KhLGKq1aLJ8S9zJ856U
 4JSdkZNDSaNzZHZKjBjDVQa+xQo+W6zA==
X-IronPort-AV: E=Sophos;i="5.90,222,1643691600"; 
   d="scan'208";a="67952087"
From: Owen Smith <owen.smith@citrix.com>
To: <win-pv-devel@lists.xenproject.org>
CC: Owen Smith <owen.smith@citrix.com>
Subject: [PATCH 2/2] Create/Destroy BlkifRing caches on Connect/Disconnect
Date: Wed, 30 Mar 2022 15:02:11 +0100
Message-ID: <20220330140211.909-2-owen.smith@citrix.com>
X-Mailer: git-send-email 2.32.0.windows.1
In-Reply-To: <20220330140211.909-1-owen.smith@citrix.com>
References: <20220330140211.909-1-owen.smith@citrix.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

Move cache creation to Connect after XENBUS_CACHE(Acquire) and cache
destruction to Disconnect before XENBUS_CACHE(Release)

Signed-off-by: Owen Smith <owen.smith@citrix.com>
---
 src/xenvbd/ring.c | 40 +++++++++++++++++++---------------------
 1 file changed, 19 insertions(+), 21 deletions(-)

diff --git a/src/xenvbd/ring.c b/src/xenvbd/ring.c
index 4482c31..ca84051 100644
--- a/src/xenvbd/ring.c
+++ b/src/xenvbd/ring.c
@@ -1898,23 +1898,14 @@ BlkifRingCreate(
 
     KeInitializeThreadedDpc(&(*BlkifRing)->Dpc, BlkifRingDpc, *BlkifRing);
 
-    status = BlkifRingCacheCreate(*BlkifRing);
-    if (!NT_SUCCESS(status))
-        goto fail4;
-
     status = ThreadCreate(RingWatchdog,
                           *BlkifRing,
                           &(*BlkifRing)->WatchdogThread);
     if (!NT_SUCCESS(status))
-        goto fail5;
+        goto fail4;
 
     return STATUS_SUCCESS;
 
-fail5:
-    Error("fail5\n");
-
-    BlkifRingCacheDestroy(*BlkifRing);
-
 fail4:
     Error("fail4\n");
 
@@ -1952,8 +1943,6 @@ BlkifRingDestroy(
     ThreadJoin(BlkifRing->WatchdogThread);
     BlkifRing->WatchdogThread = NULL;
 
-    BlkifRingCacheDestroy(BlkifRing);
-
     RtlZeroMemory(&BlkifRing->Dpc, sizeof(KDPC));
 
     ASSERT(IsListEmpty(&BlkifRing->PreparedQueue));
@@ -2003,11 +1992,15 @@ BlkifRingConnect(
     }
     KeSetImportanceDpc(&BlkifRing->Dpc, MediumHighImportance);
 
+    status = BlkifRingCacheCreate(BlkifRing);
+    if (!NT_SUCCESS(status))
+        goto fail1;
+
     BlkifRing->Mdl = __AllocatePages(1 << Ring->Order);
 
     status = STATUS_NO_MEMORY;
     if (BlkifRing->Mdl == NULL)
-        goto fail1;
+        goto fail2;
 
     BlkifRing->Shared = MmGetSystemAddressForMdlSafe(BlkifRing->Mdl,
                                                      NormalPagePriority);
@@ -2026,7 +2019,7 @@ BlkifRingConnect(
                             FALSE,
                             &BlkifRing->Grants[Index]);
         if (!NT_SUCCESS(status))
-            goto fail2;
+            goto fail3;
     }
 
     BlkifRing->Channel = XENBUS_EVTCHN(Open,
@@ -2038,7 +2031,7 @@ BlkifRingConnect(
                                        TRUE);
     status = STATUS_NO_MEMORY;
     if (BlkifRing->Channel == NULL)
-        goto fail3;
+        goto fail4;
 
     XENBUS_EVTCHN(Unmask,
                   &Ring->EvtchnInterface,
@@ -2051,7 +2044,7 @@ BlkifRingConnect(
                                  __MODULE__"|RING[%u]",
                                  BlkifRing->Index);
     if (!NT_SUCCESS(status))
-        goto fail4;
+        goto fail5;
 
     status = XENBUS_DEBUG(Register,
                           &Ring->DebugInterface,
@@ -2060,25 +2053,25 @@ BlkifRingConnect(
                           BlkifRing,
                           &BlkifRing->DebugCallback);
     if (!NT_SUCCESS(status))
-        goto fail5;
+        goto fail6;
 
     BlkifRing->Connected = TRUE;
     Trace("<==== %u\n", BlkifRing->Index);
 
     return STATUS_SUCCESS;
 
+fail6:
+    Error("fail6\n");
 fail5:
     Error("fail5\n");
-fail4:
-    Error("fail4\n");
     XENBUS_EVTCHN(Close,
                   &Ring->EvtchnInterface,
                   BlkifRing->Channel);
     BlkifRing->Channel = NULL;
+fail4:
+    Error("fail4\n");
 fail3:
     Error("fail3\n");
-fail2:
-    Error("fail2\n");
     for (Index = 0; Index < (1ul << Ring->Order); ++Index) {
         if (BlkifRing->Grants[Index] == NULL)
             continue;
@@ -2092,6 +2085,9 @@ fail2:
     __FreePages(BlkifRing->Mdl);
     BlkifRing->Shared = NULL;
     BlkifRing->Mdl = NULL;
+fail2:
+    Error("fail2\n");
+    BlkifRingCacheDestroy(BlkifRing);
 fail1:
     Error("fail1 %08x\n", status);
     return status;
@@ -2311,6 +2307,8 @@ BlkifRingDisconnect(
     BlkifRing->Shared = NULL;
     BlkifRing->Mdl = NULL;
 
+    BlkifRingCacheDestroy(BlkifRing);
+
     BlkifRing->Events = 0;
     BlkifRing->Dpcs = 0;
     BlkifRing->RequestsPosted = 0;
-- 
2.32.0.windows.1



From win-pv-devel-bounces@lists.xenproject.org Thu Mar 31 07:13:06 2022
Return-path: <win-pv-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xenproject.org
Delivery-date: Thu, 31 Mar 2022 07:13:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.296699.505117 (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nZozR-0001iF-Ob; Thu, 31 Mar 2022 07:13:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 296699.505117; Thu, 31 Mar 2022 07:13:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <win-pv-devel-bounces@lists.xenproject.org>)
	id 1nZozR-0001i8-Lc; Thu, 31 Mar 2022 07:13:05 +0000
Received: by outflank-mailman (input) for mailman id 296699;
 Thu, 31 Mar 2022 07:13:04 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qG3k=UK=citrix.com=prvs=0829e609b=owen.smith@srs-se1.protection.inumbo.net>)
 id 1nZozP-0001i2-Ly
 for win-pv-devel@lists.xenproject.org; Thu, 31 Mar 2022 07:13:03 +0000
Received: from esa3.hc3370-68.iphmx.com (esa3.hc3370-68.iphmx.com
 [216.71.145.155]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ff5eb965-b0c1-11ec-8fbc-03012f2f19d4;
 Thu, 31 Mar 2022 09:13:02 +0200 (CEST)
X-BeenThere: win-pv-devel@lists.xenproject.org
List-Id: Developer list for the Windows PV Drivers subproject
 <win-pv-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:win-pv-devel@lists.xenproject.org>
List-Help: <mailto:win-pv-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/win-pv-devel>, 
 <mailto:win-pv-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: win-pv-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "win-pv-devel" <win-pv-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ff5eb965-b0c1-11ec-8fbc-03012f2f19d4
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
  d=citrix.com; s=securemail; t=1648710781;
  h=from:to:subject:date:message-id:references:in-reply-to:
   content-transfer-encoding:mime-version;
  bh=uzo/lpaXLg2EYLIz6/BgpuFq2oIShhAi0Ws6KsI/J/8=;
  b=U5uPTqixhf2/nMqvL80/h7qhJgInCzzgmwEcBvVK0nDJ1GedubhXJPIi
   ffPDjB7GIQyaFABh5I5u2JpwFxUI8GtEFAMcRtLEgO4Y+OpqzfdIdDNvM
   PJhPdCmiC1xrwZD7la1hwNV66hY5svYZElSixuAznJqS81NgiX0NMN+71
   4=;
Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
X-SBRS: 5.1
X-MesageID: 67624281
X-Ironport-Server: esa3.hc3370-68.iphmx.com
X-Remote-IP: 162.221.156.83
X-Policy: $RELAYED
IronPort-Data: A9a23:rmEsLaxj1hzi5o/d8oZ6t+c3xirEfRIJ4+MujC+fZmUNrF6WrkVVz
 2AWWm+PM/yOMzD1edpya4nj8EgPv8OEzYA2SQZqrSAxQypGp/SeCIXCJC8cHc8zwu4v7q5Dx
 59DAjUVBJlsFhcwnj/0bv656yMUOZigHtIQMsadUsxKbVIiGX9JZS5LwbZj2NYz2oHhWmthh
 PupyyHhEA79s9JLGjp8B5Kr8HuDa9yr5Vv0FnRnDRx6lAe2e0s9VfrzFonoR5fMeaFGH/bSe
 gr25OrRElU1XfsaIojNfr7TKiXmS1NJVOSEoiI+t6OK2nCuqsGuu0qS2TV1hUp/0l20c95NJ
 Nplp5K0aRkuJ/fwp8MtYxYFFTxYA5FI9+qSSZS/mZT7I0zudnLtx7NlDV0sPJ1e8eFyaY1M3
 aVGcnZXNEnF3r/ohuLgIgVvrp1LwM3DO5wSvDd7yDDFDd4tQIzZQrWM7thdtNs1rp4RQ6yCP
 ZpAAdZpRE2HTxhuIEYJMdEjlaCmvX7kfzoF8WvA8MLb5ECMlVcsgdABKuH9Zd2MAN1L20qVu
 G/C12D4GQ0BcsySzyKf9XChjfOJmjn0MKoKHaC83u5nhhuU3GN7NfENfQLl+7/j0Bf4Ao8Bb
 RxPksYzkUQs3F6lSvnjWATinEGjji4TBtxNNdMG5g7Yn8I4/D2lLmQDSzdAbvkvu8k3WSEm2
 ze1oj/5OdB8mObLECzAr994uRv3YHFIdjFaOUfoWCNfu7HeTJcPYgUjpzqJOIq8lZXLFD752
 FhmRwBu1uxI3abnO0hWlG0rYg5ARLCUFmbZBS2NBwpJCz+Vgqb/PeREDnCBsZ59wH6xFAXpg
 ZT9s5H2ABoyJZ+MjjeRZ+4GAauk4f2IWBWF3wI/Qcd8qGr3piH+FWy13N2YDB0zWirjUWW0C
 HI/RCsLvMMDVJdURfEfj32N5zQCkvG7SIWNugH8ZdtSeJlhHDJrDwk1DXN8K1vFyRB2+YlmY
 M/zWZ/1UR4yVPQ2pBLrFrx1+eJ6mUgDKZb7GMmTI+KPiuHFOhZ4iN4tbTOzUwzOxPja+F+Mq
 YYOapPiJtc2eLSWXxQ7OLU7dDgiBXM6GYr3u4pQcOuCKRBhA2YvF7naxrZJRmCvt/09ejvgl
 p1lZnJl9Q==
IronPort-HdrOrdr: A9a23:iWlOvawlJZeFjnSFWKxwKrPxleskLtp133Aq2lEZdPULSKOlfp
 GV8MjziyWYtN9IYgBbpTnyAtj8fZq6z+853WBxB8bpYOCCggWVxe5ZnO3fKlHbak7DH6tmpN
 xdmstFeZHN5DpB/L/HCWCDer5KqrjmzEnrv5an854Ed3AtV0gK1XYdNu/vKDwQeOAwP+tcKH
 Pz3LskmxOQPVAsKuirDHgMWObO4/fRkoj9XBIADxk7rCGTkDKB8tfBYlel9yZbdwkK7aYp8G
 DDnQC8zL6kqeuHxhjV0HKWx4hKmeHm1sBICKW3+4Yow3TX+0eVjbZaKv6/VQMO0aOSAZER4Z
 zxSiIbToROArXqDyWISFXWqk7dOX0VmgPfIBej8ATeSIrCNWsH4oN69PxkWwqc5Ew6sN5m1q
 VXm2qfqppMFBvF2D/w/t7SSnhR5wOJSFcZ4JkuZkZkIP0jgX5q3P4i1VIQFI1FEDPx6YghHu
 UrBMbA5OxOeVffa3zCpGFgzNGlQ3x2R369MwM/k93Q1yITkGFyzkMeysBalnAc9IglQ50B4+
 jfKKxnmLxHU8dTZ6NgA+UKR9exFwX2MFrxGXPXJU6iGLAMOnrLpZKy6LIp5PuycJhN15c2kI
 SpaiItiYfzQTOaNSSj5uw6zvmWehTNYd3E8LAs26RE
X-IronPort-AV: E=Sophos;i="5.90,224,1643691600"; 
   d="scan'208";a="67624281"
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=cgCQ2ZBclaa79oTKfVjgKDxeqWAerdo9JiwmLDlkhe0Yfjl39ABQgHw6EWsQ2Gz+cHPf+dO2FVKzdbk8whzBqaW10ov5VpBMGfBh1OpeSIak7goUh+YsQk3DwJ+DkFl8isadf6+Q3h1Kzyb9pN0iR3uuICpaPQHWwYtGZrVfVWDnNcP4MLk75J+0JnmsBblXgcUJ9jXSmHhFZFooGVLdUAqjVQ86sAAFqfPPTdnx851oE88PlpCQKHM8E7pUsuf/HnGJHcJrrUnqbuIL/eDPBv7k2RiEXPamO3ADi5s16JgAv6w7sVkLeqxq2vkw2HkMhrNFWnH0ShxYpt+dAxoDwg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=uzo/lpaXLg2EYLIz6/BgpuFq2oIShhAi0Ws6KsI/J/8=;
 b=dD9M5owSP+jNq/J01spz2ngtVWpDJVxoUkfYYQlWnkENJdYJxNOaiIFmOiwsXFyOQfRj/b8TTdupbvINmJgYs3v+mZh6DaOm37a6DL8ej8/vT5klI5q5qBpQKbwsOYMbiynBrwKPvRXT7Z8BNTNkHrHXTKSj7BjO5Dkxj5CvL2SX1yab/cGUQos22Bv6Q7zA272fo1OJVSElAdX9cVbZk517zIuNIWQG0pg9nQWtSCIU6vMZ0/U39PnzoRXYvBDasCgQIHhMR4SJ/yqArunC3JCQnGbLs6pOG89T8qnQAMmPyDqCFJ9N84x0AnvepovpPgrESbe+sptoMxTMX9UXmw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com;
 dkim=pass header.d=citrix.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=uzo/lpaXLg2EYLIz6/BgpuFq2oIShhAi0Ws6KsI/J/8=;
 b=wnrJCgL/VMsFgJM2OMN1yXwb7jwhEpLiAPLLnuVuImp5Xd+vrbdJDWO5PuxKmo7yRZG/tOokCWpdwr4pJ9HGWeB3B8J9tCqExV+3VpqjZaZPssq1lH/A2jftvw7LjX7O12/9PG0Yu/Gcs0N+cuNdtYm/o/wKFt3L/PRATBzZS1o=
From: Owen Smith <owen.smith@citrix.com>
To: "paul@xen.org" <paul@xen.org>, "win-pv-devel@lists.xenproject.org"
	<win-pv-devel@lists.xenproject.org>
Subject: RE: [PATCH] Refactor Wmi.c
Thread-Topic: [PATCH] Refactor Wmi.c
Thread-Index: AQHYRBktznavb3WZD0a/Bmx10iJtaKzXq+kAgAAHLmCAAAG0AIABXmrg
Date: Thu, 31 Mar 2022 07:12:51 +0000
Message-ID: <DS7PR03MB54777FDCD070EF4A1CDCC41FFEE19@DS7PR03MB5477.namprd03.prod.outlook.com>
References: <20220330093223.659-1-owen.smith@citrix.com>
 <6b30173884a24369a061a9d9ba6bb595@EX13D32EUC003.ant.amazon.com>
 <DS7PR03MB54774C0D7D550C8BC5A552EEFE1F9@DS7PR03MB5477.namprd03.prod.outlook.com>
 <28abd23c-3e72-4a6a-37aa-594a9a130334@gmail.com>
In-Reply-To: <28abd23c-3e72-4a6a-37aa-594a9a130334@gmail.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: d58f8f23-7517-4a23-ba83-08da12e5de77
x-ms-traffictypediagnostic: CO2PR03MB2248:EE_
x-microsoft-antispam-prvs: <CO2PR03MB224864663228E3FA5432A9C7FEE19@CO2PR03MB2248.namprd03.prod.outlook.com>
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: ZFAco+FSp3chyautbVVacqnh7YDb6qQ9G4T+fXmXgt+p70eBmimd1845aYDdElZnI63BQJvtWpZEjjkNUdl3Dn8+dD6RDjBqV9LC3iKLlsvvovdM/8JIASow5KzP/IAESHA8Q6Ngo3VX7Fsz93WWBHnT5hyARByg1J2egoCQurBV69KS+Nr0Sa3L2xB4rL3oaywHLE6q1zyui2wZqf4u0IAX7JaTU8RSEUegcTJY0EcQRbIl39U/uI+/SRbYKn4wCi/IqFJaQMoCrPLtIiU0uEB31OtIWpfTe19dzoRzQfI2DBseanGKycUivRi+dhLK9NqdpGFbrsjpPZfj/MptRgP0HkD1xwUQcTO6SPbmXZZk5dxUu6KXhQqUvFyKe7iC+jdbbe4dyt2vWLp0CVdpwocCDUQlJvpS1Se7FuLPWeJB/XmO81U5r6xuR9Vuk5VV2+L+lXf+IBzjs825V1+PqpKMVKplkacl+t3f1EXGcZzUJxCHE3fixOyEeOLZDg+LRtK22aGDWemVzBcnvT9rEvn2h0ScHrGwLf52StZNa8dAJTttnMytalyWnLUtQLWZliIE0RMOlZp1dugWtojRRGxm0mOk5aC+zkJOgknDMc539BbXNkpfdID0HfTJHRWmwL2YJVEyNpOw8nrpj3nlk8STpT2ZyQoerZonz68INkCnVoDJzDyhdITGryKlzeEMKAbXlFPhun0NRBeGVsGE5Q==
x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5477.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(8676002)(122000001)(66446008)(66556008)(66476007)(64756008)(5660300002)(316002)(6506007)(7696005)(83380400001)(9686003)(44832011)(52536014)(26005)(53546011)(8936002)(38070700005)(66946007)(186003)(38100700002)(55016003)(76116006)(508600001)(2906002)(33656002)(71200400001)(110136005)(82960400001)(86362001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?utf-8?B?Tk1oWkd5djF3YzYvdE15YWV1STR5SWpBOG5OclgwSElHT2dIWlRRdVk1UkVq?=
 =?utf-8?B?QlJXWitKamhLRC84eGt3ZXBKdkhERENBZ1Qyb3FpaEt3NmMyWU5KYStRTDI1?=
 =?utf-8?B?N0hleEhyazVjUHdXeHdyL3AvZGdUQmNoT0ZiVlpZWStSTXBWNTczZU5xQ1NF?=
 =?utf-8?B?VHZUNzlkMjI1SEZLTVlXbHlCbStscFNjeDNQM2o5cG5CMmxzazlHdWw0bHAy?=
 =?utf-8?B?TmV5N2xYUFNlM2VlUTFpVUUwUExtVlpEditWMjJQZFVwS0ZaQkhLelFBQTEr?=
 =?utf-8?B?RktQRVRuaFdwdnZzeTVtNzJxc2ljY1hKYUFYY0NTVFVFWkNIZXArVWx4UnI2?=
 =?utf-8?B?MCtiZkxSTks2M1ZuSUhadlVESHZnQ2lvYitjNExzN2ViRHBZcDRXYkVtdnhx?=
 =?utf-8?B?Uk9oSXZvQ0ZabVkrMVZvSDEwczBURVRVK0xwREt1Z2hLUEMwN1FabDFWdXR4?=
 =?utf-8?B?eHhXcklOaXV6N2NVYUtQWkoxZ1lKK2dvaC8zYjg0aHRRVnczeEFTTVllbFVp?=
 =?utf-8?B?VHhSV3ZvdTZmS1c1a1VCRmJhK0lRVzlzQUE5VXVjVUlFTUdTaUc4YmZ1aHls?=
 =?utf-8?B?WG5TaStCWDJRQnlyeVByNHFvV3NRUERkUXg3R0JESDVWcS9BZFQxb3BVcGZU?=
 =?utf-8?B?WEZqVU40cmVUN3BscFpiMUhwL3h1MDVGWVpOb3BEb0M2MkdGWlJ6Yys0WTh2?=
 =?utf-8?B?UFkydUp2ZW5mZ2Vtdmk1NFBWOHE4bnZORk82UmphSG1tdzBOT3hJKytLd3hM?=
 =?utf-8?B?bTN3OG50a29OUmZhQ2pad1dLRnRiZUlkTnlyQUduSDlCdXlQT2ZDem5wNHZQ?=
 =?utf-8?B?SHFWZmRSL0wvbGluSmh3M1FjNTgveHZHWUdqMm9hM2JUMUErb2RjbjB2dnZM?=
 =?utf-8?B?Q2NGSDlZdkp6UWdPaVhLQXlXdDhRZ3VjUFl0bWFGbW1jdzc1OWgrYWxNbTlv?=
 =?utf-8?B?bng2OEhBN1JIcEplVTBDb2lxRk9PdE9mY00xQTN4SjMrOFRaaFZOZzlmTzdF?=
 =?utf-8?B?dzlCeVpncnFmZU04UFV1N2xjOVhJMFUxS1lBSmpKMnJkRHlSRUI5RnM3N08y?=
 =?utf-8?B?TDNJMTNENC8wRDdlbis3VTVvTWRMK3VDU084b2Zsbk9sTUJscENzUy9Bbzk0?=
 =?utf-8?B?aVBKR0ZYM0g2dk1HUzVkcWdPT1oyL25uVzhJVnZtcWU3Q0pSeG9EYmgyUEJD?=
 =?utf-8?B?VU00SVUxY0RKcndOT1NSQXc3L1BIU2VTZjZidVZWQTFpalVsWWRac0thOEtn?=
 =?utf-8?B?US9zaDFVMDdvMzlDNHB1akhMOER1M3VVNnQ5bTFja1JVMUlqUlM5Uk5udThl?=
 =?utf-8?B?Wnk5QXJ1Vk9Gc3EzeCszcTUxRE5uT3VSK0JFYTlYSWR0V0V2K2hObEY0Wk1Y?=
 =?utf-8?B?aWt4Z0ZwaktDWXAwdXYrdVNwRXJBNCtUZGdmN3kzVVIvV3hldDlvdVV3cmJj?=
 =?utf-8?B?RUxlSExjMnBOTHBKb1BuVFF1dGJFUmVHSURweVZReTR2VnFxeEZ1R1Zpc3RQ?=
 =?utf-8?B?azdyQ0VBUEQ5eVNJdXF0bW9oeHpocUVLVjZPMlFjd1hLYjRrN0pMa3FXejgv?=
 =?utf-8?B?TWlmRHFqbnFLM2NCUm5xWHptOUZFSEkwY09rL21pMStEYnlKWFFhNFFlSldq?=
 =?utf-8?B?N2VJb3prUzh3QXR3T1ZwUE13R3JVMHZTWUplRG9ORVlvSVdJMzR5dW5LUDkw?=
 =?utf-8?B?TTBTaHZQalBqbDBSQXllVHdsUy9KTldZNW1wa2ZFN2V6cmIrM0J2M0NlNVE5?=
 =?utf-8?B?VTJrdjFBdVRHNkJQOGxnRjN0dEsycUlmOG5HRzZnUHpENW90Y0ZtOGR2WjZR?=
 =?utf-8?B?K2UrV1ZSemhMcStvVVNrUE5rYWh3aHJGU1Z3MGVuMHh5YTh4ZGY2RU1QK3FN?=
 =?utf-8?B?UHVZam9SMm5GRFVaNzZWZWNuK0EwcjJKdVQvYXhlemx2VzFwb014b0RwWHdI?=
 =?utf-8?B?QStYcWRMYnpsaGdmZzNFSCt5ZkppQXZLYk83UUlkSmxIdVdnNy9kS0xnTU1F?=
 =?utf-8?B?SXNzdTVJUVZMSnFLSUlvZGpZenFCdzIybEp2ai9MQlQvREJyZjZGREVudDky?=
 =?utf-8?B?L01YeHg4STRTNVhtTmZYbVROUitxblYvR0EvQ0htMXZyRlZWMTkrR1FMZDZi?=
 =?utf-8?B?UkhRSkRUZ1paR1hDZUoySEVGUFpOTFEvZ3d6aHpPSjBLZmpJODg1dFY1WThP?=
 =?utf-8?B?bk5RaHhLdVpGZllqMjhkNDdITFpNWkQ3dllpWmZCRURaNDlocGRDajI5VUdR?=
 =?utf-8?B?Q3JkSWRRNEVqbnJtQVJBRGN5MEZUQmV5bUxOQUN5RjFxa1RSQXVCWjhIY2I2?=
 =?utf-8?B?OS9xaGppTjJGU0t5R1htbW16WW1aZlRxMFFCNU8zODRoZGg0b3BRQT09?=
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5477.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d58f8f23-7517-4a23-ba83-08da12e5de77
X-MS-Exchange-CrossTenant-originalarrivaltime: 31 Mar 2022 07:12:51.4608
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 26TFeZqGdMj4s6U/W29USivyYMw7vUdvRbzBXB9ncCue/wjUp3vSwazDDjFsDPlBWmJ3hYMv2CBC8WOTVW5tGQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO2PR03MB2248
X-OriginatorOrg: citrix.com

LS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCkZyb206IHdpbi1wdi1kZXZlbCA8d2luLXB2LWRl
dmVsLWJvdW5jZXNAbGlzdHMueGVucHJvamVjdC5vcmc+IE9uIEJlaGFsZiBPZiBEdXJyYW50LCBQ
YXVsDQpTZW50OiBXZWRuZXNkYXksIE1hcmNoIDMwLCAyMDIyIDExOjExIEFNDQpUbzogd2luLXB2
LWRldmVsQGxpc3RzLnhlbnByb2plY3Qub3JnDQpTdWJqZWN0OiBSZTogW1BBVENIXSBSZWZhY3Rv
ciBXbWkuYw0KDQpbQ0FVVElPTiAtIEVYVEVSTkFMIEVNQUlMXSBETyBOT1QgcmVwbHksIGNsaWNr
IGxpbmtzLCBvciBvcGVuIGF0dGFjaG1lbnRzIHVubGVzcyB5b3UgaGF2ZSB2ZXJpZmllZCB0aGUg
c2VuZGVyIGFuZCBrbm93IHRoZSBjb250ZW50IGlzIHNhZmUuDQoNCk9uIDMwLzAzLzIwMjIgMTE6
MDcsIE93ZW4gU21pdGggd3JvdGU6DQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZy
b206IER1cnJhbnQsIFBhdWwgPHBkdXJyYW50QGFtYXpvbi5jby51az4NCj4gU2VudDogMzAgTWFy
Y2ggMjAyMiAxMDo0MA0KPiBUbzogT3dlbiBTbWl0aCA8b3dlbi5zbWl0aEBjaXRyaXguY29tPjsg
DQo+IHdpbi1wdi1kZXZlbEBsaXN0cy54ZW5wcm9qZWN0Lm9yZw0KPiBTdWJqZWN0OiBSRTogW1BB
VENIXSBSZWZhY3RvciBXbWkuYw0KPiANCj4gW0NBVVRJT04gLSBFWFRFUk5BTCBFTUFJTF0gRE8g
Tk9UIHJlcGx5LCBjbGljayBsaW5rcywgb3Igb3BlbiBhdHRhY2htZW50cyB1bmxlc3MgeW91IGhh
dmUgdmVyaWZpZWQgdGhlIHNlbmRlciBhbmQga25vdyB0aGUgY29udGVudCBpcyBzYWZlLg0KPiAN
Cj4+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+PiBGcm9tOiB3aW4tcHYtZGV2ZWwgPHdp
bi1wdi1kZXZlbC1ib3VuY2VzQGxpc3RzLnhlbnByb2plY3Qub3JnPiBPbiANCj4+IEJlaGFsZiBP
ZiBPd2VuIFNtaXRoDQo+PiBTZW50OiAzMCBNYXJjaCAyMDIyIDEwOjMyDQo+PiBUbzogd2luLXB2
LWRldmVsQGxpc3RzLnhlbnByb2plY3Qub3JnDQo+PiBDYzogT3dlbiBTbWl0aCA8b3dlbi5zbWl0
aEBjaXRyaXguY29tPg0KPj4gU3ViamVjdDogW0VYVEVSTkFMXSBbUEFUQ0hdIFJlZmFjdG9yIFdt
aS5jDQo+Pg0KPj4gQ0FVVElPTjogVGhpcyBlbWFpbCBvcmlnaW5hdGVkIGZyb20gb3V0c2lkZSBv
ZiB0aGUgb3JnYW5pemF0aW9uLiBEbyANCj4+IG5vdCBjbGljayBsaW5rcyBvciBvcGVuIGF0dGFj
aG1lbnRzIHVubGVzcyB5b3UgY2FuIGNvbmZpcm0gdGhlIHNlbmRlciBhbmQga25vdyB0aGUgY29u
dGVudCBpcyBzYWZlLg0KPj4NCj4+DQo+Pg0KPj4gKiBNb3ZlcyBmdW5jdGlvbnMgdG8gYmUgaW4g
cmVsYXRlZCBsb2NhdGlvbnMgaW4gZmlsZQ0KPj4gKiBGb3JtYXRzIGNvZGUgdG8gYXBwcm9wcmlh
dGUgY29kZSBzdHlsZQ0KPj4gKiBJbmxpbmVzIHNvbWUgZnVuY3Rpb25zIHRoYXQgYXJlIG9ubHkg
Y2FsbGVkIGZyb20gMSBsb2NhdGlvbg0KPj4gKiBVc2VzIExJU1RfRU5UUlkgbWFjcm9zIHRvIGFj
Y2VzcyBsaW5rZWQgbGlzdHMNCj4+DQo+PiBTaWduZWQtb2ZmLWJ5OiBPd2VuIFNtaXRoIDxvd2Vu
LnNtaXRoQGNpdHJpeC5jb20+DQo+PiAtLS0NCj4+ICAgc3JjL3hlbmlmYWNlL3dtaS5jIHwgMzc5
NA0KPj4gKysrKysrKysrKysrKysrKysrKysrKy0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCj4+ICAg
MSBmaWxlIGNoYW5nZWQsIDE5MDcgaW5zZXJ0aW9ucygrKSwgMTg4NyBkZWxldGlvbnMoLSkNCj4g
DQo+IEVlay4gMzc5NCBsaW5lcyBvZiBjaGFuZ2UhIFRoYXQncyBhbG1vc3QgaW1wb3NzaWJsZSB0
byByZXZpZXcuIElzIHRoZXJlIGFueSBjaGFuY2UgeW91IGNhbiBicmVhayB0aGlzIGRvd24/DQo+
IA0KPiAgICBQYXVsDQo+IA0KPiANCj4gSXRzIG5vdCBlYXN5IHRvIGJyZWFrIGRvd24gYSByZWZh
Y3RvciBsaWtlIHRoaXMsIDEgc21hbGwgY2hhbmdlIGhhcyBtYW55IGtub2NrIG9uIGNoYW5nZXMg
dGhyb3VnaG91dCB0aGUgZmlsZS4NCj4gSXRzIHByb2JhYmx5IGVhc2llciB0byBhcHBseSBsb2Nh
bGx5IGFuZCBjb21wYXJlIHNpZGUgYnkgc2lkZSwgYnV0IEknbSANCj4gbm90IGV4cGVjdGluZyB0
aGlzIHRvIGJlIGFuIGVhc3kgb3IgcXVpY2sgcmV2aWV3IC0gSSBwcm9iYWJseSBzaG91bGQgDQo+
IGhhdmUgcG9zdGVkIDFzdCB3aXRoIFJGQw0KPiANCg0KQ2FuIHlvdSBhc3NlcnQgdGhhdCB0aGVy
ZSBpcyBubyBmdW5jdGlvbmFsIGNoYW5nZT8gSSB3aWxsIGNvbnNpZGVyIHRha2luZyBpdCBvbiB0
cnVzdCBpZiBzby4NCg0KICAgUGF1bA0KDQoNCkkndmUgbm90IG1hZGUgYW55IGZ1bmN0aW9uYWwg
Y2hhbmdlcywgZXhjZXB0IGZvciBjaGFuZ2VzIHRvIGhvdyB0aGUgTElTVF9FTlRSWSBtZW1iZXJz
IGFyZSBhY2Nlc3NlZCAtIHVzaW5nIHRoZQ0Kc3RhbmRhcmQgTElTVF9FTlRSWSBtYWNyb3MvZnVu
Y3Rpb25zIHJhdGhlciB0aGFuIGNhc3RpbmcgKHRoaXMgc2hvdWxkIG5vdCBhZmZlY3QgZnVuY3Rp
b25hbGl0eSwgYXMgdGhlIExJU1RfRU5UUlkNCml0ZW1zIGFyZSB0aGUgMXN0IGVudHJ5IGluIHRo
ZWlyIHJlc3BlY3RpdmUgc3RydWN0dXJlcykuDQoNCkkgaGF2ZSBydW4gdGhlc2UgY2hhbmdlcywg
YXMgcGFydCBvZiB0aGUgQ2l0cml4IGRyaXZlciBwYWNrYWdlICh3ZSBkb27igJl0IHBhdGNoIHRo
aXMgZmlsZSwgYnV0IGRvIHJlbmFtZS9yZS1HVUlEIHRoZSBXTUkNCm9iamVjdHMpLCB0aHJvdWdo
IGF1dG9tYXRlZCB0ZXN0aW5nIHdoaWNoIHZhbGlkYXRlcyB0aGUgV01JIGludGVyZmFjZXMuDQoN
ClRoaXMgcGF0Y2ggd2FzIG9yaWdpbmFsbHkgdG8gY2xlYW4gdXAgdGhlIGNvZGUgd2l0aCB0aGUg
aW50ZW50IHRvIHRyYWNrIGRvd24gYSAweDEzOSAoS0VSTkVMX1NFQ1VSSVRZX0NIRUNLX0ZBSUxV
UkUpDQpidWdjaGVjaywgdGhhdCBpbmRpY2F0ZWQgdGhlcmUgd2FzIG9jY2FzaW9uYWwgTElTVF9F
TlRSWSBjb3JydXB0aW9uLiBBZnRlciB0aGVzZSBjaGFuZ2VzLCBJIGhhdmUgbm90IHlldCByZXBy
b2R1Y2VkDQp0aGUgYnVnY2hlY2sNCg0KT3dlbg0K


